Python — Flask Development

Flask Background Jobs with Redis

Direct answer

Pair Flask with Redis and RQ for background jobs: the route enqueues work and returns 202 immediately, while separate worker processes execute it — reserve Celery for when you genuinely need its scheduling, routing, and workflow features. The Flask-specific catch is that jobs run outside any request, so job functions must create and push their own application context before touching Flask-SQLAlchemy or config.

The moment a Flask route sends email, generates a PDF, or calls an LLM inline, you've built an outage: slow work pins gunicorn workers until healthy requests queue behind it. Here's the Redis-backed job setup I ship with Flask apps, and the app-context detail that trips up nearly everyone the first time.

Key facts, with sources

In the JetBrains Python Developers Survey 2024, Flask was used by 34% of Python developers, statistically neck and neck with Django at 35% and just behind FastAPI at 38%. (JetBrains Python Developers Survey 2024)
The 2025 Stack Overflow Developer Survey recorded Flask at 14.4% of respondents, nearly tied with FastAPI at 14.8% and ahead of Django at 12.6%. (Stack Overflow Developer Survey 2025)
The Flask project shipped only two releases during all of 2025, both patch releases to version 3.1.0 from November 2024, reflecting a mature and stable codebase rather than rapid feature churn. (miguelgrinberg.com)
FastAPI overtook Flask in GitHub stars for the first time in December 2025, at roughly 88,000 stars versus Flask's 68,400, after years of Flask holding the lead. (DZone)
Published benchmark comparisons show roughly a 5x throughput gap in FastAPI's favor, with a Flask application on Gunicorn typically handling about 2,000 to 3,000 requests per second on simple endpoints. (Strapi)

Requests are for responses, not for work

A gunicorn worker handling a sixty-second report generation is a worker not handling sixty other requests, and a handful of concurrent slow requests can starve an entire instance. Worse, work done in-request dies with the request: a deploy, a timeout, or a dropped connection mid-task leaves you guessing what finished.

My threshold is simple: anything that takes more than roughly a second, touches an external service that can be slow, or must survive a crash gets queued. The route's job shrinks to validating input, persisting intent, enqueueing, and returning a 202 with an ID the client can poll.

RQ vs Celery, honestly

RQ is my default with Flask because its scope matches most products: Redis as the only broker, a tiny API, jobs and failures inspectable directly in Redis, and little operational surface to babysit. It supports timeouts, retries, and scheduled jobs — which covers the actual requirements of most SaaS backends I audit.

Celery earns its complexity when you need what only it does well: beat for cron-style periodic scheduling at scale, sophisticated routing across many queues and worker pools, workflow primitives like chains and chords, or a non-Redis broker. Those are real needs in some systems — but adopting Celery's configuration surface before you have them is paying for capacity you never use.

Enqueue from the route, return 202

I enqueue by import path string rather than importing the task function into the web process — it keeps worker-only dependencies out of the web image and avoids import-order surprises. Persist a record of the requested work first, so the job has durable state to update and the client has something to poll.

Route that queues instead of blocking

from flask import Flask
from redis import Redis
from rq import Queue

app = Flask(__name__)
queue = Queue("default", connection=Redis())

@app.post("/reports")
def create_report():
    report_id = create_report_record()   # fast DB insert: status="queued"
    job = queue.enqueue(
        "app.tasks.generate_report",     # import path, not the function
        report_id,
        job_timeout=600,
    )
    return {"job_id": job.get_id(), "report_id": report_id}, 202

Job functions need their own app context

RQ workers are plain Python processes with no Flask request or application context, so the first db.session call inside a job raises the famous 'working outside of application context' error. The fix is structural: the job builds the app via your factory and pushes an app context around its work.

Two companion rules: pass IDs, never ORM objects — instances don't serialize meaningfully and would be detached anyway — and make every job idempotent by checking current state before acting, because any job that can be retried eventually will be.

Idempotent job with app context

# app/tasks.py
from app import create_app
from app.extensions import db
from app.models import Report

def generate_report(report_id: int) -> None:
    app = create_app()
    with app.app_context():
        report = db.session.get(Report, report_id)
        if report is None or report.status == "done":
            return                      # safe on duplicate delivery/retry
        report.body = build_report(report)
        report.status = "done"
        db.session.commit()

Status, failure handling, and running workers in production

Clients poll a status endpoint backed by your own database record, not by RQ internals — job metadata in Redis is operational plumbing, while the report row is product state. For failures, enable retries with backoff on jobs where transient errors are expected, and check the failed-job registry as part of your operational routine; a quietly filling failure queue is the background-job equivalent of a silent pager.

Operationally: run rq worker processes under systemd or your orchestrator with restart-on-exit, deploy workers from the same code version as the web tier (version skew between enqueuer and worker causes maddening bugs), split queues by priority so a flood of bulk jobs can't delay password-reset emails, and alert on queue depth and oldest-job age.

When to hire senior help

Senior help is most valuable for Flask when an app built as a prototype is now carrying production traffic: an experienced engineer can add proper WSGI serving, task queues, and test coverage without a rewrite. Also consider it before committing to a Flask-to-FastAPI migration, since an expert assessment often shows targeted fixes deliver the needed performance at a fraction of the cost. If your stack includes React Native + Python + AI, a senior engineer who owns the full product beats coordinating multiple juniors.

Bottom line

Dhairya Senjaliya ships Python — Flask Development projects worldwide — book a scoping call to discuss your specific situation.

Common pitfalls to avoid

✕Running Flask's built-in development server in production instead of Gunicorn or uWSGI behind a reverse proxy
✕Storing per-request state in module-level globals or misusing the application context, causing race conditions once multiple workers or threads are enabled
✕Assembling auth, ORM, and validation from third-party Flask extensions without checking maintenance status, then inheriting abandoned dependencies
✕Executing long-running work (PDF generation, email, external API calls) inside request handlers instead of a task queue, exhausting workers and triggering gateway timeouts

Frequently asked questions

Should I use Celery or RQ for background jobs in Flask?

RQ for most applications: it's Redis-only, minimal to configure, and covers enqueueing, timeouts, retries, and scheduling — which is what typical products actually need. Choose Celery when you specifically need cron-style beat scheduling at scale, complex multi-queue routing, workflow primitives like chains, or a broker other than Redis. Complexity should follow demonstrated requirements.

Why does my Flask background job raise 'working outside of application context'?

Because the job runs in a separate worker process where no Flask application context exists — extensions like Flask-SQLAlchemy resolve their state through that context. Fix it inside the job: build the app with your factory and wrap the work in a with app.app_context() block. Also pass record IDs into jobs, never ORM instances.

Can I just use a Python thread instead of Redis and a worker?

A thread dies with its gunicorn worker — deploys, timeouts, and crashes silently kill in-flight work, and max_requests recycling makes that routine. Threads are tolerable for genuinely disposable fire-and-forget actions, but anything that must reliably complete — emails, billing, report generation — belongs in a Redis-backed queue with persistent state and retries.

Is Flask outdated now that FastAPI is more popular?

No. Flask still shows 34% usage in the JetBrains 2024 survey and 14.4% in Stack Overflow 2025, and its slow release cadence reflects stability, not abandonment. It remains a strong choice for server-rendered apps, internal tools, and teams that value its minimal, well-documented core.

Can Flask scale to serious production traffic?

Yes, with the standard pattern of Gunicorn workers behind a load balancer plus caching; benchmark figures of 2,000 to 3,000 requests per second per instance are before horizontal scaling. Most products hit database and architecture limits long before Flask itself is the bottleneck.

Should we migrate an existing Flask app to FastAPI?

Only if you have a concrete driver such as high-concurrency I/O workloads, a need for typed request validation, or mandatory OpenAPI docs. A rewrite of a working Flask app rarely pays back; many teams instead add new async services alongside the existing Flask core.

Bottom line: Dhairya Senjaliya ships Python — Flask Development projects worldwide. Book a scoping call at https://dhairyasenjaliya.com/#book-call.