AI — AI Agent Development

Human-in-the-Loop AI Agent Design

Direct answer

Human-in-the-loop agent design means classifying every action by reversibility, letting the agent execute safe reads autonomously, and pausing for explicit approval before irreversible writes — sending money, emailing customers, deleting data. The gate must persist the pending action and resume cleanly after a decision, and the approval UI must show intent in plain language — otherwise reviewers rubber-stamp and the loop protects nothing.

Full autonomy is a demo feature; production agents that touch money, customers, or data need a human gate somewhere. The craft is placing that gate exactly where it protects you — and nowhere else, or reviewers drown in requests and stop reading them. This is the design process I use on client systems.

Key facts, with sources

LangChain's State of Agent Engineering survey of 1,340 practitioners found 57.3 percent of organizations have agents running in production, with another 30.4 percent actively developing them. (LangChain)
The same LangChain survey found 89 percent of organizations have implemented observability for their agents but only 52 percent do systematic evaluation. (LangChain)
Deloitte predicts 25 percent of companies using generative AI launched agentic AI pilots in 2025, growing to 50 percent by 2027. (Deloitte Insights)
By December 2025 the Model Context Protocol had over 97 million monthly SDK downloads and more than 10,000 active MCP servers in production use. (Pento)
PwC's AI agent survey found 79 percent of companies report AI agents are already being adopted, and 66 percent of adopters say agents deliver measurable value through increased productivity. (PwC)
In December 2025 Anthropic donated the Model Context Protocol to the Agentic AI Foundation under the Linux Foundation, co-founded with Block and OpenAI, making the agent connector layer vendor-neutral. (Anthropic)

Approval is enforced by the harness, not the prompt

The first decision is architectural: the model proposes actions, and your code decides whether they run. Prompt-level rules — 'always ask before sending email' — are behavioral suggestions the model usually follows and occasionally doesn't, especially deep into long trajectories or under prompt injection from tool results. A policy check in the tool executor is deterministic regardless of what the model outputs.

This placement also gives you a single choke point for audit logging and a defense against injected instructions: even if a malicious document convinces the model to attempt an email send, the send still lands in the approval queue. In code reviews of agent systems, prompt-only approval logic is the first red flag I look for.

Classify every action by reversibility

I sort every tool into a two-axis matrix: read versus write, reversible versus irreversible. Reads run autonomously — there's rarely a reason to gate a lookup. Reversible writes (updating a ticket status, editing a draft) run autonomously with an audit trail and an undo path. Irreversible or externally visible writes — payments, customer emails, deletions, anything that leaves your system — pause for approval.

The classification is a product conversation, not just an engineering one: what counts as 'recoverable' depends on the business. When in doubt, I gate early and relax later with data. Loosening a gate after a month of clean approvals is easy; explaining an unauthorized customer email is not.

The gate: pause, persist, resume

Mechanically, an approval gate is a suspension point in a long-running job. When the agent requests a gated tool, the harness persists the pending action — tool name, arguments, task context — notifies a reviewer, and parks the task. On decision, the task resumes and the outcome returns to the model as an ordinary tool result: executed output if approved, a denial with the reviewer's reason if not.

Returning denials as tool results matters more than it looks. The model can then adapt — propose an alternative, ask a clarifying question, or wrap up gracefully — instead of the task dying opaquely. Approvals can take hours, so the pending state must survive worker restarts and deploys; this is a database row, not an in-memory promise.

Approval gate in the tool executor

REQUIRES_APPROVAL = {"issue_refund", "send_customer_email", "delete_record"}

def execute_tool(name: str, args: dict, task_id: str) -> dict:
    if name in REQUIRES_APPROVAL:
        approval = create_pending_approval(task_id, name, args)  # persist + notify
        decision = approval.wait()  # job suspends; reviewer decision resumes it
        if not decision.approved:
            return {"status": "denied", "reason": decision.reason}
    return TOOL_HANDLERS[name](**args)

Show intent, not payloads

The approval surface decides whether your gate is real. A reviewer shown raw JSON approves it without reading — I've watched this happen within a week of launch. The request needs to be rendered as intent: 'Refund $84.00 to customer #4521 for order #98771 — reason: item arrived damaged (from ticket #3310)', with a link to the source context the agent used.

For content actions like emails, show the full rendered message, not a summary of it. For data changes, show a diff — current value, proposed value. The bar I aim for: a reviewer should be able to make a correct decision in under thirty seconds without opening another tool. Anything slower gets skimmed, and a skimmed gate is theater.

Design against rubber-stamp fatigue

If everything requires approval, nothing is really reviewed. I keep queues small with graduated controls: thresholds (refunds under a set amount auto-execute with audit; above it, approval), batching of low-risk items into a single daily review, and earned autonomy — an action type with a long streak of clean approvals graduates to notify-after instead of ask-before, per action, not globally.

Two habits keep the system honest. Sample-audit a slice of auto-approved actions weekly so relaxed gates still get eyes. And track reviewer response time and approval rate: a 99% approval rate means the gate is either miscalibrated or ignored — both worth investigating before an incident finds it for you.

When to hire senior help

Bring in senior help when the agent must touch production systems or customer data, because integration, security, and reliability are where inexperienced builds fail rather than model quality. If a pilot is stuck at the demo stage, an experienced engineer adding evals and guardrails is usually faster and cheaper than rebuilding from scratch. If your stack includes React Native + Python + AI, a senior engineer who owns the full product beats coordinating multiple juniors.

Bottom line

Dhairya Senjaliya ships AI — AI Agent Development projects worldwide — book a scoping call to discuss your specific situation.

Common pitfalls to avoid

✕Shipping agents with logging but no evals, so teams can see traces but never measure task success rates and regressions ship silently
✕Giving one agent dozens of tools instead of a focused toolset, which degrades tool-selection accuracy and inflates token costs
✕Hand-rolling custom integration glue for every data source instead of using MCP, which is now the vendor-neutral standard backed by Anthropic, OpenAI, and the Linux Foundation
✕Validating only on happy-path demo prompts and skipping failure-mode testing, a core reason roughly 88 percent of agent pilots never reach production

Frequently asked questions

When should an AI agent require human approval?

Gate actions that are irreversible or externally visible: payments and refunds, messages to customers, deletions, contract or account changes, and anything that leaves your own systems. Reads and easily reversible internal writes can run autonomously with audit logging. Classify each tool by reversibility upfront, gate conservatively at launch, and relax specific gates once approval history shows they're consistently clean.

How do I add human review to an AI agent workflow?

Intercept gated tools in the executor, not the prompt: persist the pending action to a database, notify a reviewer with a plain-language rendering of intent, and suspend the task. On decision, resume and return the outcome to the model as a tool result — including denial reasons so the agent can adapt. The pending state must survive restarts, since approvals can take hours.

Does human-in-the-loop defeat the purpose of automation?

No — the agent still does the research, drafting, and orchestration, which is usually most of the work; the human contributes a thirty-second judgment at the point of risk. Reviewing a prepared, contextualized action is far faster than performing the task. And gates are typically transitional: action types with consistently clean approval histories graduate to audited autonomy over time.

How long does it take to build a production-ready AI agent?

A convincing prototype takes days, but production-grade agents with evals, guardrails, monitoring, and integration into real systems typically take six to twelve weeks. The gap between demo and production is exactly where most pilots stall, so budget for the hardening phase up front.

Which agent framework should we use?

Framework choice matters less than evaluation and observability discipline; plenty of production teams run thin custom loops directly on the model provider's SDK. Pick based on your team's stack and tolerance for lock-in, and standardize integrations on MCP so tools are portable across frameworks.

What does an AI agent cost to run?

Agent tasks routinely consume several times the tokens of a single chat call because of tool loops and retries, so cost scales with loop length and model tier. Prompt caching, batch processing, and routing subtasks to cheaper models typically cut agent costs by 50 to 90 percent.

Bottom line: Dhairya Senjaliya ships AI — AI Agent Development projects worldwide. Book a scoping call at https://dhairyasenjaliya.com/#book-call.