OpenAI API Security: Key Management and Proxying

Key facts, with sources

• At DevDay 2025 OpenAI reported 800 million weekly ChatGPT users, 4 million developers building on its platform, and roughly 8 billion API tokens processed per minute. (CNBC)
• ChatGPT reached 900 million weekly active users by late February 2026, up from 800 million at DevDay in October 2025. (TechCrunch)
• By March 2026 OpenAI's APIs were processing more than 15 billion tokens per minute, roughly doubling from the rate reported at DevDay 2025. (Panto AI OpenAI Statistics)
• OpenAI's published API pricing discounts cached input tokens by 90 percent on supported GPT models, which materially cuts costs for agents that resend long system prompts. (OpenAI API Pricing Docs)
• OpenAI raised $122 billion in new funding in 2026 to accelerate the next phase of AI development, one of the largest private raises in history. (OpenAI)

The threat model in practice

Three leak paths cover nearly every real incident. Client-side exposure: a key shipped in a mobile or web bundle is extractable by anyone with the app and a proxy tool — assume it is public the moment it ships. Repository leaks: a key committed to git, even briefly and even in a private repo, persists in history and eventually escapes through forks, CI logs, or a repo going public. Operational leaks: keys echoed into application logs, error trackers, or build output where far more people and systems can read them than intended.

The blast radius is twofold: direct spend by whoever holds the key, and data exposure, since the key authorizes sending content to the API. A fourth, subtler risk is your own proxy — an unauthenticated internal AI endpoint is a free LLM gateway for anyone who finds it, and scanners do find them.

Key hygiene that actually holds up

Keys belong in a secrets manager and reach processes as runtime environment configuration — never hardcoded, never in .env files that get committed, never in client-side variables (anything prefixed for public exposure in web or Expo builds ships to users). Use projects to scope keys narrowly: one key per service per environment, so a compromised staging key cannot spend production budget, and set per-project spend limits so the worst case is bounded.

Rotation should be routine, not an emergency skill. If rotating a key requires a deploy and a prayer, you will hesitate during a real incident. Wire secret scanning into CI and pre-commit hooks so a pasted key never lands in history — the scan costs seconds and has caught real keys in most teams I have set it up for.

The proxy is your enforcement point

Clients — mobile apps, browsers, third-party integrations — never hold provider keys. They authenticate to your backend, and your backend talks to OpenAI. That proxy is where policy lives: user authentication, per-user and per-tier quotas, output caps, usage metering, and redaction. It is also your abstraction seam for switching providers or adding fallbacks later.

import os
from fastapi import FastAPI, Depends, HTTPException
from openai import AsyncOpenAI

MODEL = os.environ["OPENAI_MODEL"]  # set to the latest model id
app = FastAPI()
client = AsyncOpenAI()  # key injected from the secret manager at deploy time

@app.post("/v1/generate")
async def generate(body: GenerateRequest, user=Depends(require_user)):
    if not await quota.allow(user.id, estimated_tokens(body)):
        raise HTTPException(429, "AI quota exceeded for your plan")
    clean_input = redact_pii(body.text)  # strip emails, card numbers, etc.
    resp = await client.chat.completions.create(
        model=MODEL,
        messages=[{"role": "system", "content": SYSTEM_PROMPT},
                  {"role": "user", "content": clean_input}],
    )
    await meter.record(user.id, resp.usage.total_tokens)
    return {"output": resp.choices[0].message.content}

Protecting data in prompts and logs

Whatever your users type becomes an API payload, so decide deliberately what may leave your infrastructure. Redact obvious sensitive classes — payment numbers, government identifiers, credentials — before requests go out; regex catches the structured ones cheaply, and stricter products add an NER pass. Understand your provider's data handling terms for API traffic, and where contracts require it, pursue the stricter retention arrangements enterprise agreements can offer rather than assuming defaults.

Your own logging is often the bigger leak. Full prompt logging is invaluable for debugging and evals, but it duplicates user data into systems with wider access. Log prompt-template versions and token counts everywhere; store raw prompt content only where you have a governed store with access controls and a retention window. Error trackers deserve scrubbing rules too — stack traces love to capture request bodies.

Detection and incident response

Assume a leak will eventually happen and optimize for noticing quickly. Watch usage dashboards per key and per project; alert on spend velocity, unfamiliar model usage, and traffic at hours your product does not have. A stolen key typically announces itself as a usage pattern your product could not produce — sustained throughput, odd models, no corresponding application traffic.

Write the runbook before you need it: rotate the affected key, confirm services picked up the new secret, review usage logs to bound the exposure window, and check whether prompt content in that window included user data that triggers disclosure obligations. With per-service scoped keys and spend caps already in place, this is an annoying afternoon; with one shared key across everything and no caps, the same event is a budget crisis plus a full-system credential rotation.

When to hire senior help

Bring in senior help when you move from a working prototype to production traffic, because cost controls, evals, rate-limit handling, and fallback behavior determine whether the unit economics work. An experienced engineer usually pays for themselves by cutting token spend and preventing outages rather than by writing the first prompt. If your stack includes React Native + Python + AI, a senior engineer who owns the full product beats coordinating multiple juniors.

Bottom line

Dhairya Senjaliya ships AI — OpenAI Development projects worldwide — book a scoping call to discuss your specific situation.

Common pitfalls to avoid

• ✕Hardcoding a single flagship model ID for every call instead of routing by task, paying GPT-5-tier prices for classification work a nano-tier model handles at a fraction of the cost
• ✕Putting volatile content like timestamps and user IDs at the top of prompts, which breaks prefix caching and forfeits the 90 percent cached-input discount
• ✕Building on deprecated surfaces like the legacy Completions or wound-down fine-tuning APIs instead of the current Responses API and agent tooling
• ✕Launching with no spend caps or per-user rate limits, so a retry loop or a single abusive user burns a month's API budget overnight

Frequently asked questions

Sources

• CNBC
• TechCrunch
• Panto AI OpenAI Statistics
• OpenAI API Pricing Docs
• OpenAI