Multi-Agent Systems: Orchestrator Pattern

Key facts, with sources

• Anthropic reported that a multi-agent research system using an Opus lead agent with Sonnet subagents outperformed a single-agent Opus baseline by 90.2 percent on its internal research eval. (ByteByteGo)
• Anthropic's multi-agent research system used about 15x more tokens than a normal chat interaction, and token usage alone explained roughly 80 percent of performance variance. (The AI Engineer)
• The MAST research taxonomy identified 14 distinct failure modes across 7 popular multi-agent frameworks including AutoGen, ChatDev, and CrewAI, grouped into system design flaws, inter-agent misalignment, and task verification failures. (arXiv)
• Salesforce research found organizations run an average of 12 AI agents and projects multi-agent adoption to surge 67 percent within two years as enterprises move toward orchestration. (Salesforce)
• Multi-agent orchestration with three or more agents represents about 22 percent of enterprise agent deployments in 2026, projected to reach roughly 45 to 50 percent by 2027. (OnAbout AI)

What the orchestrator actually owns

The orchestrator does four jobs: it decomposes the incoming task into subtasks, decides which worker handles each one, collects and validates the results, and decides when the overall task is done. Workers are deliberately ignorant of the big picture — each one sees only its subtask and the minimum context needed to complete it.

This is different from a pipeline, where output flows linearly from one agent to the next, and from peer-to-peer designs, where agents message each other directly. Pipelines are fine for fixed sequences; peer-to-peer looks impressive in demos and becomes untraceable in production. The orchestrator sits in the middle: dynamic routing, but every decision passes through one component you can log, test, and reason about.

Why centralized control wins in production

Every operational concern gets exactly one home. Token budgets: the orchestrator counts spend across all workers and halts the run at a ceiling. Retries: a failed subtask retries at the orchestrator without re-running everything upstream. Tracing: one component emits the full task tree, so any run can be reconstructed after the fact.

The less obvious win is failure isolation. When a worker produces garbage, the orchestrator validates the result against the subtask contract and can re-dispatch — to the same worker with error feedback, or to a different one. In decentralized designs, a bad output propagates through two or three agents before anything notices, and by then the entire run is poisoned.

A minimal implementation

I implement workers as plain functions wrapping one LLM call with a role-specific system prompt. The orchestrator is a strong model prompted to plan: it emits a JSON list of subtasks, each naming a worker and an instruction, and a plain loop dispatches them. Resist the urge to make the loop itself an agent — deterministic dispatch code is far easier to test than a model deciding control flow token by token.

In real systems I add three things to this skeleton: schema validation on every worker result, a per-run token budget checked before each dispatch, and structured logging of every handoff. But the shape stays exactly this small.

import json
import anthropic

MODEL = "claude-sonnet-latest"  # replace with the latest Claude model id
client = anthropic.Anthropic()

WORKERS = {
    "researcher": "You extract key facts from the provided material. Return bullet points only.",
    "writer": "You turn research notes into clear prose for a technical audience.",
    "critic": "You review a draft and list concrete problems. No rewrites, just findings.",
}

def run_worker(name: str, instruction: str, context: str) -> str:
    resp = client.messages.create(
        model=MODEL,
        max_tokens=2000,
        system=WORKERS[name],
        messages=[{"role": "user", "content": f"Task: {instruction}\n\nContext:\n{context}"}],
    )
    return resp.content[0].text

def orchestrate(task: str) -> str:
    plan = client.messages.create(
        model=MODEL,
        max_tokens=1000,
        system="Decompose the task into subtasks. Return only JSON: "
               '[{"worker": "researcher|writer|critic", "instruction": "..."}]',
        messages=[{"role": "user", "content": task}],
    )
    context = ""
    for step in json.loads(plan.content[0].text):
        context = run_worker(step["worker"], step["instruction"], context)
    return context

Handoffs are contracts, not conversations

The biggest quality lever is not the prompts — it is what passes between agents. I define a typed result schema per worker: the researcher returns a list of facts with source references, the critic returns findings with severity levels. The orchestrator validates each result before it becomes input to the next dispatch, so a malformed output fails immediately at the boundary instead of quietly degrading three steps later.

I also trim context aggressively per dispatch. Workers get the subtask instruction plus only the artifacts they need — never the full transcript of everything that came before. This keeps token cost roughly linear in the number of subtasks instead of quadratic, and it stops workers from getting distracted by irrelevant history.

Where the pattern breaks down

The orchestrator becomes a bottleneck in two ways. Contextually: on long runs its own window fills with plans and intermediate results, so you need summarization or externalized state to keep it lean. Structurally: hub-and-spoke adds a round trip per handoff, so latency-sensitive workflows with many tiny steps suffer.

It is also the wrong shape when two workers genuinely need to iterate together — a generator and a checker going back and forth several times should not route every exchange through the hub. There I let the pair run as a nested loop that the orchestrator treats as one worker. And when the task is a fixed three-step sequence, skip the pattern entirely: a pipeline of plain function calls is simpler and just as capable.

When to hire senior help

Multi-agent orchestration is one of the least commoditized skills in AI engineering, and teams that succeed usually include someone who has debugged coordination failures in production. Get senior review before committing to an orchestrator-worker design, because architectural mistakes at this layer are expensive to unwind after launch. If your stack includes React Native + Python + AI, a senior engineer who owns the full product beats coordinating multiple juniors.

Bottom line

Dhairya Senjaliya ships AI — Multi-Agent Architectures projects worldwide — book a scoping call to discuss your specific situation.

Common pitfalls to avoid

• ✕Defaulting to multi-agent when a single agent with good tools would do, since the roughly 15x token multiplier only pays off when subtasks are genuinely parallel and high value
• ✕Letting subagents share full conversation history instead of scoped task briefs, causing context bloat, contradictory actions, and coordination failures
• ✕Shipping without a verification layer, so errors propagate through agent chains unchecked; task verification failures are one of the three MAST failure categories
• ✕Skipping per-agent trace observability, which makes it impossible to identify which agent in the chain caused a bad final output

Frequently asked questions

Sources

• ByteByteGo
• The AI Engineer
• arXiv
• Salesforce
• OnAbout AI