BackgroundTasks vs Celery — Decision Matrix

FastAPI’s BackgroundTasks and Celery both let you run work “after the response is sent”. They look similar; they solve different problems. Mixing them up is a common production mistake.

What FastAPI BackgroundTasks does

from fastapi import BackgroundTasks

@app.post("/users")
async def create_user(user: UserIn, bg: BackgroundTasks):
    new_user = await db.create_user(user)
    bg.add_task(send_welcome_email, new_user.email)
    return new_user

After the response is returned, FastAPI runs send_welcome_email in the same process. That’s it. No queue. No retries. No persistence.

If the process crashes between response-send and task-run, the task is lost. If it raises, FastAPI logs and moves on — no retry, no DLQ.

What Celery does

@celery_app.task(bind=True, max_retries=5, autoretry_for=(SMTPError,), retry_backoff=True)
def send_welcome_email(self, email):
    smtp.send(email)

@app.post("/users")
async def create_user(user: UserIn):
    new_user = await db.create_user(user)
    send_welcome_email.delay(new_user.email)
    return new_user

delay() enqueues a message to the broker (Redis / RabbitMQ / SQS). A separate worker process picks it up, runs it, retries on failure, dead-letters on exhaustion. Crash-resilient.

Decision matrix

When BackgroundTasks is fine

• Sending a single email after signup. Fire-and-forget — if it fails, retry isn’t critical (or you have a retry layer in your email provider).
• Logging / metrics emission. Best-effort observability.
• Invalidating a cache key. Stale-tolerant.
• Pre-warming something. Best-effort.

Rule of thumb: if losing the task is acceptable, BackgroundTasks works.

When you need Celery (or equivalent)

• Payment processing, billing, financial transactions. Cannot lose.
• Anything with retries. “Send this even if the SMTP server is flaky.”
• Long-running work. Image transcoding, batch jobs, ML inference.
• Scheduled tasks. Daily reports, hourly cleanups.
• Workflows. Multi-step processes with branching, compensation (consider Temporal).
• Work that scales independently of the API. Worker fleet sized to backlog.

Alternatives to Celery

For an async FastAPI app, arq is a nice fit — same asyncio ecosystem, Redis-based, simple API.

Hybrid: BackgroundTasks → Celery

A pattern: use BackgroundTasks to enqueue to Celery, so the API response doesn’t wait on the broker round-trip even:

async def enqueue_email(email):
    send_welcome_email.delay(email)  # very fast but technically blocking

@app.post("/users")
async def create_user(user: UserIn, bg: BackgroundTasks):
    new_user = await db.create_user(user)
    bg.add_task(enqueue_email, new_user.email)
    return new_user

Probably overkill for a single .delay() call (it’s already fast), but useful if the enqueue path is heavier.

How BackgroundTasks runs

FastAPI / Starlette runs background tasks in the same event loop after the response is returned. For sync def tasks, runs them in the threadpool (same as for sync routes).

A long-running BackgroundTask will hold up the response cleanup but does not block other concurrent requests — they run on the loop concurrently. But the process must stay alive until the task finishes; in K8s, terminationGracePeriodSeconds must cover it.

Reliability in BackgroundTasks: zero

@app.post("/charge")
async def charge(req: ChargeIn, bg: BackgroundTasks):
    await stripe.charge(...)         # commits the money
    bg.add_task(notify_finance, req)  # if this never runs, finance never knows
    return {"ok": True}

Process crashes between the response and notify_finance running → finance is silently out of sync. This is not acceptable for anything load-bearing.

The right shape: write the side effect to your DB in the same transaction as the business change, then a separate process picks it up reliably. Transactional outbox pattern — see 13_architecture_design/15_event_driven_saga.md or the new outbox file.

When to keep BackgroundTasks long-term

Even in a Celery-heavy app, BackgroundTasks remains useful for:

• Response observability — emit a metric after the response.
• Logging that doesn’t fit in middleware — domain-specific audit log line.
• In-process cache priming — warm something the next request will want.

The thing they all share: losing them is fine.

Interview angle

• “BackgroundTasks vs Celery — when each?” — BackgroundTasks: in-process, no persistence, no retries, no broker. Use for fire-and-forget where loss is acceptable. Celery: durable queue, retries, distributed workers, scheduled tasks. Use for anything that must not be lost.
• “You have a 30-second video transcoding step after upload. BackgroundTasks?” — no. Too long for an in-process task (ties the worker, no crash recovery, no retry, no isolation). Push to Celery / Fargate task / Lambda + S3 trigger.
• “What happens if the process crashes mid-BackgroundTask?” — task is lost silently. No retry, no DLQ. For anything where loss matters, use a real queue.
• “Are there async-native alternatives to Celery for FastAPI?” — arq (Redis, async-native, simpler API), dramatiq, RQ (sync). For workflows (multi-step, long-running, sagas), Temporal. For AWS-native, SQS + Lambda.
• “How would you implement a ‘send email after signup’ that must not be lost?” — write the email job to your DB inside the user-creation transaction (outbox pattern), or push to Celery / SQS. Don’t rely on BackgroundTasks.
• “What’s the latency cost of Celery vs BackgroundTasks?” — BackgroundTasks runs in-process, sub-millisecond. Celery has broker round-trip (~ms on Redis local, more on cross-region brokers). For most user-facing flows the broker round-trip is invisible.