Error Handling in Multi-Step AI Workflows

Key facts, with sources

• McKinsey's State of AI 2025 found nearly nine in ten organizations now use AI in at least one business function, yet only about 6 percent attribute 5 percent or more of EBIT to their AI use. (McKinsey)
• McKinsey found AI high performers are 2.8x more likely than others to have fundamentally redesigned workflows (55 percent versus 20 percent), and workflow redesign has the biggest effect on realizing EBIT impact from gen AI. (McKinsey)
• Zapier's survey of 525 enterprise executives found human-in-the-loop is the most common agent management approach at 38 percent, while 20 percent say their AI systems now operate autonomously with minimal oversight. (Zapier)
• 84 percent of enterprise leaders say they will likely or certainly increase AI agent investment over the next 12 months, with customer support (49 percent) and operations (47 percent) leading deployment. (Yahoo Finance)
• Menlo Ventures found coding and developer tools were the largest enterprise AI workflow category at $7.3 billion in 2025 spend, with half of developers now using AI tools daily. (Menlo Ventures)

Three failure classes, three different responses

The root mistake I find in audits is one catch-all handler treating every exception identically — retrying what can never succeed and giving up on what would have succeeded in two seconds. AI workflows fail in three distinct ways. Transient failures come from infrastructure: provider rate limits, overload responses, network timeouts, momentary downstream outages. These deserve automatic retries with exponential backoff. Quality failures are the AI-specific class: the call succeeded but the output is wrong — invalid JSON, a hallucinated field, a refusal, content that fails validation. Retrying identically sometimes helps; a targeted repair prompt helps more often. Permanent failures — malformed input, authorization errors, an unsupported document type — will never succeed on retry and should route straight to a dead-letter queue or review lane.

Every handler in the pipeline should decide which class it is looking at before deciding what to do. That single discipline drives everything else here.

Retrying transient errors properly

For transient failures I use exponential backoff with jitter, honoring the provider's retry-after signal when present, and I retry only the exception types that are genuinely transient. Catching broadly here is actively harmful: retrying a validation error five times burns tokens to fail five times.

import anthropic
from tenacity import (
    retry,
    retry_if_exception_type,
    stop_after_attempt,
    wait_exponential_jitter,
)

RETRYABLE = (
    anthropic.RateLimitError,        # 429 — back off and retry
    anthropic.InternalServerError,   # 5xx — provider-side, retryable
    anthropic.APIConnectionError,    # network failure before a response
)

@retry(
    retry=retry_if_exception_type(RETRYABLE),
    wait=wait_exponential_jitter(initial=1, max=30),
    stop=stop_after_attempt(5),
    reraise=True,
)
def call_step(client: anthropic.Anthropic, **request):
    return client.messages.create(**request)

Checkpoint so failures resume, not restart

In a five-step pipeline where each step is a paid model call, re-running the whole workflow because step four failed multiplies your costs by the failure rate. The fix is checkpointing: persist each step's output as soon as it completes, keyed by run id, and make the runner skip any step whose result already exists. A retried run then flows straight to the failure point with prior results intact.

This also requires steps to be idempotent — re-executing a completed step must not duplicate side effects — so downstream writes should be upserts keyed by run id, and model outputs cached by input hash.

def run_workflow(run_id: str, steps: list, store) -> dict:
    state = store.load(run_id) or {}
    for step in steps:
        if step.name in state:
            continue  # completed on a previous attempt — skip, don't re-pay
        state[step.name] = step.execute(state)
        store.save(run_id, state)  # checkpoint after every step
    return state

Treat bad output as a first-class error

The failure class unique to AI workflows is the successful call with unusable content. Handle it deliberately: validate every model output against a schema and business rules immediately, and on failure run a bounded repair loop — send the invalid output back with the specific validation errors and ask for a correction. One repair attempt resolves a large share of quality failures in my experience; more than two attempts rarely pays, so cap the loop and escalate.

Critically, log quality failures with the offending output and the validation reasons. Aggregated, these logs are your prompt-improvement backlog: recurring failure patterns tell you exactly which instruction or schema tweak would eliminate a whole error class, which beats handling it at runtime forever.

Dead-letter queues and alerting that respects your sleep

Whatever survives retries and repair goes to a dead-letter queue with everything needed to diagnose it: full input, step outputs so far, every error encountered, and timing. A DLQ turns mystery failures into a reviewable work queue — and in AI pipelines it doubles as a goldmine, because clustered DLQ entries reveal systematic issues like a document layout your extraction prompt cannot handle.

Alert on rates and trends, not single failures: a workflow with retries and a DLQ is designed to absorb individual errors silently. Page on sustained failure-rate spikes, DLQ growth, or a stalled queue — those indicate something structural like an expired credential or a provider incident. Track error rates per step and per failure class on a dashboard; that breakdown is what tells you whether this week's problem is infrastructure, prompts, or inputs.

When to hire senior help

Bring in senior help when workflows cross system boundaries such as CRM, billing, or anything touching customer PII, or when a no-code prototype hits reliability and cost limits. The redesign work itself, mapping the process, defining checkpoints, and instrumenting metrics, benefits most from someone who has shipped production AI workflows before. If your stack includes React Native + Python + AI, a senior engineer who owns the full product beats coordinating multiple juniors.

Bottom line

Dhairya Senjaliya ships AI — AI Workflows projects worldwide — book a scoping call to discuss your specific situation.

Common pitfalls to avoid

• ✕Bolting AI onto an existing process instead of redesigning it, when McKinsey data shows redesign, not adoption, separates the roughly 6 percent of companies seeing real EBIT impact
• ✕Automating a workflow nobody measured first, leaving no baseline to prove time or cost savings when budget review comes
• ✕Using an expensive frontier model for every step instead of routing simple steps to cheap models and reserving reasoning models for the hard ones
• ✕Jumping to full autonomy on day one and skipping the human-in-the-loop stage most enterprises use to build trust and surface failure modes

Frequently asked questions

Sources

• McKinsey
• McKinsey
• Zapier
• Yahoo Finance
• Menlo Ventures