Specialist Agents for Legal Document Review

Key facts, with sources

• Anthropic reported that a multi-agent research system using an Opus lead agent with Sonnet subagents outperformed a single-agent Opus baseline by 90.2 percent on its internal research eval. (ByteByteGo)
• Anthropic's multi-agent research system used about 15x more tokens than a normal chat interaction, and token usage alone explained roughly 80 percent of performance variance. (The AI Engineer)
• The MAST research taxonomy identified 14 distinct failure modes across 7 popular multi-agent frameworks including AutoGen, ChatDev, and CrewAI, grouped into system design flaws, inter-agent misalignment, and task verification failures. (arXiv)
• Salesforce research found organizations run an average of 12 AI agents and projects multi-agent adoption to surge 67 percent within two years as enterprises move toward orchestration. (Salesforce)
• Multi-agent orchestration with three or more agents represents about 22 percent of enterprise agent deployments in 2026, projected to reach roughly 45 to 50 percent by 2027. (OnAbout AI)

Why one mega-prompt fails on contracts

A single agent asked to 'review this agreement' faces three compounding problems. Contracts are long, and attention over a hundred pages of definitions, cross-references, and exhibits is exactly where models degrade — the subtle indemnity carve-out on page forty gets summarized away. The tasks conflict: extraction wants exhaustive literalism, risk assessment wants judgment, summarization wants compression, and one prompt optimizing for all three does each worse. And a single free-form review is unverifiable — the lawyer cannot tell which claims are grounded without re-reading the document, which defeats the purpose.

Splitting into specialists fixes all three: each agent gets a narrow rubric, a scoped slice of the document, and a structured output schema that can be checked mechanically.

The specialist lineup

My standard lineup has four roles. The extractor pulls structured facts — parties, term, renewal mechanics, payment obligations, termination triggers, governing law — into a fixed schema. It makes no judgments; it only locates and quotes. The risk flagger compares each clause category against a playbook of preferred and unacceptable positions — the client's, not a generic one — and flags deviations with severity. The consistency checker hunts for internal defects: defined terms used but never defined, cross-references pointing at renumbered sections, exhibit lists that do not match attached exhibits. The memo writer assembles everything into a triaged review memo, ordered by severity.

Each specialist runs with its own retrieval scope and prompt, and the orchestrator owns sequencing: extraction first, because the risk and consistency agents consume its structured output rather than re-reading the raw document.

Citations are the product

The rule I enforce without exception: every finding carries a verbatim quote of the clause it refers to, plus its location. A flag that says 'the limitation of liability appears one-sided' is noise; a flag that quotes the sentence, names the section, and states which playbook position it violates is a work product a lawyer can verify in seconds.

This rule also becomes a mechanical hallucination filter. Because quotes must be verbatim, I check every cited string against the source document before a finding reaches the memo — a plain string match, no model involved. Findings whose quotes do not appear in the document are dropped and logged. This single check eliminates the most damaging failure class in legal AI: confident commentary about clauses that do not exist.

Confidence, escalation, and the human boundary

Each specialist labels findings with a confidence level, and the thresholds have teeth: low-confidence findings route to a separate 'needs human read' queue rather than being silently included or silently dropped. Both failure directions are bad — silent inclusion erodes trust, silent dropping hides risk — so ambiguity gets surfaced as ambiguity.

The system's scope also stops deliberately short of advice. Agents identify, quote, and compare against the playbook; they do not recommend whether to sign. That boundary is not modesty, it is product design: the tool's job is to make a lawyer's first pass dramatically faster and more complete, and tools that overstep into judgment get banned by the same lawyers who would otherwise champion them.

Evaluating against a golden set

Before any deployment, I build a golden set: real contracts of the target type, annotated by a lawyer with every finding the system should have produced. Each specialist is then measured separately — extraction accuracy for the extractor, precision and recall per risk category for the flagger. Aggregate scores hide exactly the failures that matter, like a flagger that is excellent on payment terms and blind on IP assignment.

Precision deserves special attention because false positives are the adoption killer. A lawyer who wades through invented risks a few times stops reading the memo entirely. I would rather ship a system that catches fewer issues with high precision and expand its recall category by category than ship a noisy one that gets ignored.

Confidentiality and audit obligations

Legal documents carry constraints that most AI products never face. The model provider agreement needs scrutiny: no training on submitted content, defined retention windows, and data processing terms the firm's clients would accept. Prompts and intermediate agent outputs are themselves sensitive — they contain contract text — so logs and traces need the same access controls and retention policies as the documents.

An audit trail is non-negotiable: which document version, which agent, which prompt version, which model produced each finding, and who reviewed it. Privilege questions — whether and how AI-assisted review interacts with legal professional privilege — vary by jurisdiction and are for the firm's own counsel to settle before rollout, not after. I build the trail assuming someone will eventually have to reconstruct exactly what the system saw and said.

When to hire senior help

Multi-agent orchestration is one of the least commoditized skills in AI engineering, and teams that succeed usually include someone who has debugged coordination failures in production. Get senior review before committing to an orchestrator-worker design, because architectural mistakes at this layer are expensive to unwind after launch. If your stack includes React Native + Python + AI, a senior engineer who owns the full product beats coordinating multiple juniors.

Bottom line

Dhairya Senjaliya ships AI — Multi-Agent Architectures projects worldwide — book a scoping call to discuss your specific situation.

Common pitfalls to avoid

• ✕Defaulting to multi-agent when a single agent with good tools would do, since the roughly 15x token multiplier only pays off when subtasks are genuinely parallel and high value
• ✕Letting subagents share full conversation history instead of scoped task briefs, causing context bloat, contradictory actions, and coordination failures
• ✕Shipping without a verification layer, so errors propagate through agent chains unchecked; task verification failures are one of the three MAST failure categories
• ✕Skipping per-agent trace observability, which makes it impossible to identify which agent in the chain caused a bad final output

Frequently asked questions

Sources

• ByteByteGo
• The AI Engineer
• arXiv
• Salesforce
• OnAbout AI