Claude Computer Use: Early Production Lessons

Key facts, with sources

• Anthropic's published API pricing lists Claude Sonnet 4.6 at $3 per million input tokens and $15 per million output tokens, with Claude Haiku 4.5 at $1 and $5 for lighter workloads. (Claude Platform Docs)
• The Claude API offers a 50 percent discount on both input and output tokens via the Batch API and up to 90 percent savings on repeated input through prompt caching. (Claude Platform Docs)
• Current Claude Opus and Sonnet models support a 1 million token context window at flat per-token rates with no long-context surcharge. (CloudZero)
• Anthropic raised a $30 billion Series G at a $380 billion post-money valuation in 2026. (Anthropic)
• Anthropic said it hit a $30 billion revenue run rate after roughly 80x growth in about two years, driven primarily by enterprise and developer API consumption. (VentureBeat)

What computer use actually is

Computer use puts Claude in an observe-act loop with a real screen: your harness captures a screenshot, sends it to the model, receives an action — click these coordinates, type this text, press these keys, scroll — executes it against a real desktop environment you host, then screenshots again. The model reasons visually about what is on screen, which means it can operate software that has no API, no DOM to script against, and no automation hooks whatsoever.

That is also the cost structure: every step is a model round trip carrying an image, so a task that takes a human twelve clicks becomes a dozen-plus vision-model calls executed sequentially. Understanding computer use as expensive, general, and slow — versus API integration as cheap, narrow, and fast — is the framing every subsequent decision hangs on.

Where it earned its keep

Three categories have justified the cost in my work. First, legacy systems without APIs — the ERP module from another decade, the government portal, the vendor dashboard with no export. Where the alternative is a human copy-pasting for hours or a six-month integration negotiation, an agent that fills forms from structured data wins even at a much higher per-task cost than an API call. Second, cross-application glue: workflows spanning three tools where each hop is trivial but the combination never justified real integration work. Third, UI smoke testing — a natural-language agent checking that critical flows still work catches whole classes of visual and layout breakage that selector-based test scripts sail past, and it does not shatter when a class name changes.

The common thread: low-to-moderate volume, high manual cost, no API alternative. Computer use is a bridge over integration gaps, not a replacement for integrations you could actually build.

Where it struggled

Honest failure list. Latency: screenshot-reason-act cycles stack up, so multi-step tasks run minutes, not seconds — fine for background jobs, wrong for anything a user watches. Dynamic interfaces: toasts, lazy-loaded content, layout shifts, and unexpected modals break naive action sequences; the agent clicks where a button was a moment ago. Precision targets: dense tables, tiny icons, and drag interactions still produce wrong-element clicks at rates that matter. Cost: image-heavy loops consume tokens quickly, and a retried flaky task doubles its own bill.

And drift: applications update their UIs on their own schedule, so a flow verified last month can degrade silently. Anything running unsupervised needs outcome verification built in — check the record actually saved, the export actually downloaded — because with computer use, silent failure is the default failure mode.

The guardrails that make it production-safe

An agent with mouse and keyboard on a real system is a security surface, and I treat it like one. It runs in a disposable, sandboxed VM containing only the target applications, with network egress restricted to what the task requires — never on a machine holding credentials or data beyond the task's scope. It operates a dedicated account with minimum permissions, so the blast radius of a wrong click is bounded by design rather than by hope.

Irreversible actions get checkpoints: the agent stages the work — fills the form, prepares the submission — and a human approves the final commit, asynchronously in batches so the checkpoint does not erase the labor savings. Every step is logged with its screenshot, which makes failures diagnosable and gives you an audit trail. Prompt injection is real here too: screen content is untrusted input, and text on a webpage can try to instruct the agent, which is one more reason scope stays narrow and confirmations stay human.

Decide with a simple hierarchy

My integration decision tree, in order: if an API exists, use the API — computer use is never the right choice when a stable programmatic interface is available. If there is no API but the volume is high and the target is stable, consider conventional scripted automation; it is cheaper per run once built, though far more brittle to UI change. Computer use takes the remainder: no API, moderate volume, changing or complex UIs, or tasks where the model's judgment during the flow — interpreting an unexpected screen, deciding which record matches — is itself the value.

Budget honestly across the pilot: model tokens per completed task including retries, VM infrastructure, and the human review slice. Some pilots will show the economics beat the manual process they replace; others will show a part-time contractor is still cheaper. Both are useful answers, and the capability is improving fast enough that the losing cases deserve a re-run each model generation.

When to hire senior help

Senior help matters most when you go beyond simple completions into agentic systems on the Claude API, where tool design, caching architecture, and eval harnesses decide reliability and cost. A few days of experienced review typically cuts token bills materially and prevents expensive rewrites later. If your stack includes React Native + Python + AI, a senior engineer who owns the full product beats coordinating multiple juniors.

Bottom line

Dhairya Senjaliya ships AI — Claude API Development projects worldwide — book a scoping call to discuss your specific situation.

Common pitfalls to avoid

• ✕Skipping prompt caching in agent loops that resend the same system prompt and tool definitions every turn, paying full input price for tokens that could cost 90 percent less
• ✕Running latency-insensitive workloads like evals, backfills, and bulk classification through the live API instead of the Batch API's 50 percent discount
• ✕Migrating model versions by swapping the ID string without checking for removed parameters like temperature or thinking budgets, which now return 400 errors on newer Claude models
• ✕Treating refusal and max-token stop reasons as generic errors instead of branching on stop_reason, which surfaces as silent empty responses in production

Frequently asked questions

Sources

• Claude Platform Docs
• CloudZero
• Anthropic
• VentureBeat