---
title: "AI Managing AI"
description: "The most capable AI models now spend their cycles orchestrating other tools. Specialization creates startup opportunity : build the specialists the executives call first."
categories: ["AI","agents","infrastructure"]
keywords: ["AI agents","model orchestration","tool calling","bitter lesson","distillation","SOTA models","AI infrastructure","specialized agents","frontier models","function calling"]
ai_summary: "Frontier AI models are becoming managers, orchestrating specialized agents rather than doing tasks directly. As tool calling accuracy crosses 90%, startups can build niche specialists that plug into these constellations."
date: 2026-01-22
lastmod: 2026-07-23
canonical_url: https://www.tomtunguz.com/the-scaffolding-shift/
author: "Tomasz Tunguz"
---


Talented people get promoted to management. So do talented models. Claude manages code execution. Gemini routes requests across CRM & chat. GPT-5 can coordinate public stock research.

Why now? Tool calling accuracy crossed a threshold. Two years ago, GPT-4 succeeded on fewer than 50% of function-calling tasks. Models hallucinated parameters, called wrong endpoints, forgot context mid-conversation. Today, SOTA models exceed 90% accuracy on function-calling benchmarks[^1]. Performance of the most recent models, like [Gemini 3](/gemini-3-proves-pretraining-scaling-laws-intact/), is materially better in practice than the benchmarks suggest.

{{< email_image src="bnubgckktqef9vjd5srr" alt="Tool calling accuracy over time" width="540" height="304" >}}

Did we need trillion-parameter models just to make function calls? Surprisingly, yes.

Experiments with small action models, lightweight networks trained only for tool selection, fail in production[^2]. They lack world knowledge. Management, it turns out, requires context.

Today, the orchestrator often spawns itself as a subagent (Claude Code spins up another Claude Code). This symmetry won't last.

The bitter lesson[^3] insists ever-larger models should handle everything. But economics push back : distillation & reinforcement fine-tuning produce models 40% smaller & 60% faster while retaining 97% of performance[^4].

Specialized agents from different vendors are emerging. The frontier model becomes the executive, routing requests across specialists. These specialists can be third-party vendors, all vying to be best in their domain.

{{< email_image src="syxu4qcisx7kl9lu2wve" alt="Constellation of Models architecture" width="400" height="604" >}}

Constellations of specialists require reliable tool calling. When tool calling works 50% of the time, teams build monoliths, keeping everything inside one model to minimize failure points. When it works 90% of the time, teams route to specialists & compound their capabilities.

The frontier labs will own the orchestration layer. But they can't own every specialist. Startups that build the best browser-use agent, the best retrieval system, the best BI agent can plug into these constellations & own their niche.

New startup opportunities emerge not from training the largest models, but from training the specialists the executives call first.

[^1]: Berkeley Function Calling Leaderboard ([BFCL](https://gorilla.cs.berkeley.edu/leaderboard.html)) tests API invocation accuracy. TAU-bench measures tool-augmented reasoning in real-world scenarios ([paper](https://arxiv.org/abs/2406.12045)).

[^2]: Salesforce's [xLAM](https://tomtunguz.com/local-instructions/) is a large action model designed specifically for tool selection. While fast & accurate for simple tool calls, small action models struggle with complex reasoning about when to use tools.

[^3]: Rich Sutton's influential essay arguing that general methods leveraging computation beat hand-engineered domain knowledge. [The Bitter Lesson](http://www.incompleteideas.net/IncIdeas/BitterLesson.html).

[^4]: See [DistilBERT](https://arxiv.org/abs/1910.01108), which is 40% smaller & 60% faster while retaining 97% of BERT's performance. OpenAI's [model distillation](https://platform.openai.com/docs/guides/distillation) enables similar efficiency gains.