AI Workflow Orchestration with Temporal

Key facts, with sources

• McKinsey's State of AI 2025 found nearly nine in ten organizations now use AI in at least one business function, yet only about 6 percent attribute 5 percent or more of EBIT to their AI use. (McKinsey)
• McKinsey found AI high performers are 2.8x more likely than others to have fundamentally redesigned workflows (55 percent versus 20 percent), and workflow redesign has the biggest effect on realizing EBIT impact from gen AI. (McKinsey)
• Zapier's survey of 525 enterprise executives found human-in-the-loop is the most common agent management approach at 38 percent, while 20 percent say their AI systems now operate autonomously with minimal oversight. (Zapier)
• 84 percent of enterprise leaders say they will likely or certainly increase AI agent investment over the next 12 months, with customer support (49 percent) and operations (47 percent) leading deployment. (Yahoo Finance)
• Menlo Ventures found coding and developer tools were the largest enterprise AI workflow category at $7.3 billion in 2025 spend, with half of developers now using AI tools daily. (Menlo Ventures)

Why AI pipelines need durable execution

A five-step document pipeline — classify, extract, enrich, validate, write back — has failure surfaces at every step: model rate limits, malformed outputs, flaky third-party APIs, and your own deploys restarting workers mid-run. Hand-rolled solutions accrete the same pieces every time: a state table, checkpoint writes, a reconciliation cron for stuck runs, and retry logic sprinkled through worker code.

Temporal replaces all of that with event-sourced workflow state. When a worker dies at step four, another worker picks up the workflow and resumes exactly there — steps one through three are not re-executed, their recorded results are replayed from history. For pipelines where each step costs real money in tokens, not re-running completed work is the headline feature.

Workflows orchestrate, activities do the work

Temporal's one non-negotiable rule shapes the whole design: workflow code must be deterministic, because it gets replayed from history to reconstruct state. That means no network calls, no random values, no wall-clock reads inside the workflow function. Every LLM call, embedding lookup, database write, and HTTP request belongs in an activity.

This constraint turns out to be a good forcing function. Your workflow file becomes a readable description of the business process — the ordering, branching, and compensation logic — while each activity is a small, independently testable function with explicit inputs and outputs. In code audits, Temporal-based AI pipelines are consistently the easiest to reason about, because the orchestration is not smeared across queue handlers.

A document pipeline in Temporal Python

This is the minimal shape I start from: activities own the model calls, the workflow sequences them with per-step timeouts and retry policies. Temporal handles persistence, retries, and resumption; the code contains none of that machinery.

from datetime import timedelta
from temporalio import activity, workflow
from temporalio.common import RetryPolicy

@activity.defn
async def classify_document(doc_id: str) -> str:
    # LLM call lives here, never in the workflow function
    ...

@activity.defn
async def extract_fields(doc_id: str, doc_type: str) -> dict:
    ...

@workflow.defn
class DocumentPipeline:
    @workflow.run
    async def run(self, doc_id: str) -> dict:
        doc_type = await workflow.execute_activity(
            classify_document,
            doc_id,
            start_to_close_timeout=timedelta(minutes=2),
            retry_policy=RetryPolicy(maximum_attempts=5, backoff_coefficient=2.0),
        )
        return await workflow.execute_activity(
            extract_fields,
            args=[doc_id, doc_type],
            start_to_close_timeout=timedelta(minutes=5),
            retry_policy=RetryPolicy(maximum_attempts=5),
        )

Tuning retries for LLM realities

Default retry policies treat all failures the same, which wastes money on AI workloads. I separate three cases. Transient provider errors — rate limits, overloads, timeouts — should retry with backoff, and Temporal's per-activity policies handle that cleanly. Validation failures, where the model returned schema-breaking output, deserve a small bounded number of retries because a fresh attempt often succeeds. Permanent failures — an unsupported document type, an authorization error — should be raised as non-retryable application errors so the workflow can branch to a human-review path instead of burning five attempts.

For long-running activities, set heartbeats so Temporal detects a hung worker in seconds rather than waiting out the full timeout. And keep activity inputs and outputs small — pass document ids and store payloads in object storage, since workflow histories carry every payload.

When Temporal is the wrong choice

Temporal earns its operational cost when workflows are multi-step, long-lived, or expensive to re-run. It is the wrong tool for single model calls behind an endpoint — a plain worker with retries is simpler — and for teams unwilling to run or pay for the orchestration layer, since self-hosting the cluster is real infrastructure work and the managed option is a real line item.

The middle ground I use as a rule of thumb: fewer than three steps and under a minute of runtime, use a queue and a worker; anything with fan-out, human-in-the-loop waits, multi-day timers, or compensation logic, use Temporal. Migrating later is feasible — activities are just functions — so starting simple and graduating when the workflow grows teeth is a legitimate path.

When to hire senior help

Bring in senior help when workflows cross system boundaries such as CRM, billing, or anything touching customer PII, or when a no-code prototype hits reliability and cost limits. The redesign work itself, mapping the process, defining checkpoints, and instrumenting metrics, benefits most from someone who has shipped production AI workflows before. If your stack includes React Native + Python + AI, a senior engineer who owns the full product beats coordinating multiple juniors.

Bottom line

Dhairya Senjaliya ships AI — AI Workflows projects worldwide — book a scoping call to discuss your specific situation.

Common pitfalls to avoid

• ✕Bolting AI onto an existing process instead of redesigning it, when McKinsey data shows redesign, not adoption, separates the roughly 6 percent of companies seeing real EBIT impact
• ✕Automating a workflow nobody measured first, leaving no baseline to prove time or cost savings when budget review comes
• ✕Using an expensive frontier model for every step instead of routing simple steps to cheap models and reserving reasoning models for the hard ones
• ✕Jumping to full autonomy on day one and skipping the human-in-the-loop stage most enterprises use to build trust and surface failure modes

Frequently asked questions

Sources

• McKinsey
• McKinsey
• Zapier
• Yahoo Finance
• Menlo Ventures