Compliance Considerations for Agentic Systems

Key facts, with sources

• Gartner predicts over 40 percent of agentic AI projects will be canceled by the end of 2027 due to escalating costs, unclear business value, or inadequate risk controls. (Gartner)
• Gartner predicts 33 percent of enterprise software applications will include agentic AI by 2028, up from less than 1 percent in 2024. (Gartner)
• Gartner estimates only about 130 of the thousands of vendors claiming to sell agentic AI are real, with the rest engaged in agent washing of existing chatbots and RPA products. (MarTech)
• McKinsey's State of AI 2025 found 23 percent of organizations are scaling an agentic AI system somewhere in the enterprise and another 39 percent have begun experimenting with agents. (McKinsey)
• Gartner forecasts 40 percent of enterprise applications will embed task-specific AI agents by the end of 2026, up from under 5 percent in 2025. (Joget)

Actions change the compliance question

Traditional AI compliance asks what the model said and whether it was appropriate. Agentic compliance asks what the system did, under whose authority, and who is accountable — the questions you would ask about an employee. That reframing is useful: for each action the agent can take, identify the control that would apply if a junior staff member did it, and implement the equivalent. If a human refunding a payment needs a second approval above a threshold, the agent does too. If a human accessing sensitive records leaves an access-log entry, so does the agent.

This mapping exercise, done early, produces your tool permission design almost automatically — and it gives auditors a framing they already understand instead of a novel AI story they have to reason about from scratch.

The audit trail is your most important control

Every regulated conversation I have had about agents eventually arrives at the same artifact: the run log. It needs to capture, immutably, who initiated the run, the prompt and model version, every tool call with inputs and outputs, every approval with the approver's identity and timestamp, and the final outcome. Store it append-only with restricted access, retained on a defined schedule.

The standard to aim for is reconstructibility: months later, you can replay exactly what the agent saw and did, step by step, and explain why an action occurred. That single capability answers a remarkable share of auditor questions across SOC 2, GDPR accountability, and sector rules — and it is dramatically cheaper to build into the loop from day one than to retrofit under deadline after the first serious customer asks.

Data protection: minimize what enters the context

Agents are data-hungry by default — the easy implementation stuffs everything potentially relevant into the context — and that is precisely the wrong instinct in regulated environments. Apply minimization at the tool boundary: tools return the fields the task needs, not whole records; identifiers are masked or pseudonymized where the model does not need them; and free-text fields from users, which hide PII in unpredictable places, get scrubbed or filtered before entering the context.

Remember the transcript is itself a data store. Whatever enters the context ends up in your run logs, so retention and deletion obligations follow it there. A subject-erasure request under GDPR-style rules reaches into stored transcripts too — design your log schema so per-subject deletion or redaction is a query, not an archaeology project.

Human oversight: define it, don't imply it

Regulatory frameworks increasingly expect meaningful human oversight of automated decisions with significant effects — and "a human could theoretically review the logs" does not qualify. Meaningful oversight means defined checkpoints: which action categories require pre-approval, who is authorized to approve, what information the approver sees, and what happens on rejection. Write it down as policy, enforce it in the tool layer, and record every approval in the audit trail.

Guard against rubber-stamping, because auditors and plaintiffs both look for it. If your approval data shows long streaks of near-instant decisions, that is documentation of oversight theater. I design approval UIs to render the full consequence of the action and track review time as a metric — partly for UX, mostly because it makes the oversight defensible.

Your model provider is a subprocessor

Your model provider processes whatever data your agent's context contains, which makes them a subprocessor in most privacy frameworks. That triggers real work: a data processing agreement with the provider, listing them in your subprocessor register, notifying customers whose contracts require it, and understanding the provider's retention behavior and training-use terms — including whether reduced-retention options exist and what features they exclude.

Region routing matters for data-residency commitments: know where inference actually runs, not just where your servers are. And check your customer contracts in the other direction — many enterprise agreements contain clauses about automated decision-making or AI use written years ago that your agentic feature may technically trip. Legal review before launch is much cheaper than a breach-of-contract conversation after.

Prepare for the incident before it happens

Agents fail in novel ways — prompt injection steering a tool call, an approval bypass bug, a runaway loop mass-modifying records — and your incident response plan needs agent-specific playbooks the same way it has database-breach playbooks. Pre-answer the mechanics: how to freeze all agent activity immediately with a tested kill switch, how to identify affected runs from the audit trail, how to reverse reversible actions in bulk, and what notification duties trigger if personal data was involved.

Run a tabletop exercise before launch with a concrete scenario. Teams discover in the exercise that revocation is slow, logs are missing a field they need, or nobody owns the freeze decision — all cheap to fix before an incident and expensive during one. The exercise also generates exactly the evidence of preparedness that enterprise security reviews ask for.

When to hire senior help

Senior help is most valuable at the architecture stage, deciding what to automate, where approval gates belong, and how business value will be measured, before any code is written. It is also worth bringing in when a stalled pilot needs risk controls and evaluation rigor to pass security and compliance review. If your stack includes React Native + Python + AI, a senior engineer who owns the full product beats coordinating multiple juniors.

Bottom line

Dhairya Senjaliya ships AI — Agentic AI Systems projects worldwide — book a scoping call to discuss your specific situation.

Common pitfalls to avoid

• ✕Buying agent-washed products, since Gartner estimates only around 130 of thousands of self-described agentic AI vendors are genuine rather than rebranded chatbots or RPA
• ✕Deploying autonomy before defining risk controls and human-approval gates, one of the three causes Gartner cites for the 40 percent of projects it expects to be canceled
• ✕Measuring activity like tasks attempted instead of business value, leaving the project unable to justify escalating costs at renewal time
• ✕Wrapping agents around existing processes instead of redesigning the workflow, when McKinsey finds workflow redesign is the single biggest driver of EBIT impact from gen AI

Frequently asked questions

Sources

• Gartner
• MarTech
• McKinsey
• Joget