Practical Case: LLM File Processing Pipeline with Daily Report

Scenario

Users upload 10–20 files per day to a system. Each file must be processed through an LLM and the extracted data saved to a database. A daily report on processed files needs to be generated. Can we use the cloud?

This is a small-volume, batch-oriented pipeline. Don’t over-engineer it. The interesting parts are: where files live, how processing is triggered, how to handle LLM failures, and what “daily report” actually means.

How would you design this system?

The flow has four stages:

1. Upload — user pushes a file to object storage (S3 / GCS / Azure Blob) via a pre-signed URL, or through your API which then writes to storage.
2. Trigger — a message lands on a queue (or a storage event fires) saying “process this file.”
3. Process — a worker picks up the message, downloads the file, sends content to the LLM, validates the structured output, writes to DB.
4. Report — a scheduled job runs once per day, queries the DB for the last 24h, formats a report, delivers it (email / Slack / dashboard).

At 10–20 files/day, this is roughly one file every 1–2 hours. You don’t need autoscaling, sharding, or a streaming platform. A single worker is enough; the design questions are durability and retries, not throughput.

[user] -> [API or pre-signed URL] -> [object storage]
                                          |
                                          v (event/notification)
                                       [queue]
                                          |
                                          v
                                      [worker]
                                       |    |
                              [LLM API]    [DB]
                                          ^
                                          |
                                  [scheduler] -- daily --> [report -> email/Slack]

Can it run in the cloud? Which services?

Yes, and at this volume cloud is usually cheaper and simpler than self-hosting. Three roughly equivalent stacks:

For this volume, serverless (Lambda / Cloud Run / Functions) is the best fit: zero idle cost, no servers to patch, native triggers from object storage. One function triggered by an S3 ObjectCreated event, writing to RDS, plus an EventBridge rule for the daily report — that’s the whole system.

If the team already runs Kubernetes, a Deployment + a CronJob in the existing cluster works just as well. Pick what’s already operated.

What does the worker actually do?

def handle_file(s3_key: str) -> None:
    file_id = extract_file_id(s3_key)

    if already_processed(file_id):           # idempotency
        return

    raw = download(s3_key)
    text = extract_text(raw)                 # PDF/docx/image -> text

    result = call_llm_with_retry(
        prompt=PROMPT,
        text=text,
        response_schema=ExtractedData,       # pydantic model
    )

    save_to_db(file_id, result, status="ok")

Key points:

• Idempotency. Storage events can fire twice. Check file_id before processing or use an upsert. Make file_id the natural key.
• Structured output. Force the LLM to return JSON matching a Pydantic schema. Reject and retry on schema violations rather than saving bad rows.
• Retries. Wrap the LLM call in retry-with-backoff. Distinguish 429 (back off longer) from 5xx (retry) from 4xx (don’t retry, log and dead-letter).
• Dead-letter queue. After N retries, push to a DLQ so you can inspect failures without blocking the pipeline.
• Cost cap. A runaway loop calling an LLM API burns money fast. Set a per-file token cap and a daily budget alarm.

How do you generate the daily report?

A scheduled job (EventBridge / Cloud Scheduler / Celery Beat / cron) runs once per day:

def daily_report():
    rows = db.query(
        "SELECT * FROM processed_files "
        "WHERE processed_at >= NOW() - INTERVAL '1 day'"
    )
    summary = {
        "total": len(rows),
        "ok": sum(r.status == "ok" for r in rows),
        "failed": sum(r.status == "failed" for r in rows),
        "by_category": Counter(r.category for r in rows),
    }
    send_email(to=REPORT_RECIPIENTS, body=render(summary, rows))

Pick a delivery channel that matches the audience: email or Slack for humans, BI tool (Metabase / Superset) reading the DB directly if analysts already use one. For 10–20 rows/day, don’t build a dashboard — send a Slack message.

Run the report in the same runtime as the worker (same Lambda, different handler; same container image, different command). Don’t spin up a separate service for one query a day.

What about cost?

Rough monthly cost at ~20 files/day, ~10K input + 1K output tokens per call:

• Object storage: cents.
• Queue: covered by free tier.
• Serverless worker: ~600 invocations/month — covered by free tier.
• Database: smallest managed Postgres ≈ $15/mo, or serverless option (Aurora Serverless v2, Neon, Supabase free tier).
• LLM API: dominates the bill. ~600 calls/month on a small/cheap model is a couple of dollars; on a frontier model it’s one to two orders of magnitude more. Quote the ratio, not a price — per-token pricing moves every few months.
• The 2026 default is model routing: classify the document first, send the easy 80% to a small model, escalate only the ambiguous ones to a frontier model. Saying this unprompted is what separates a current answer from a 2023 one.

Total: under $50/mo on a small model, under $100/mo even on a frontier model. Cloud infra is not the expensive part — the LLM is.

What can go wrong?

• LLM is non-deterministic. Same input can produce different output. If the report needs reproducibility, store the raw LLM response alongside the parsed fields.
• Schema drift. Prompt or model change can break parsing silently. Validate with Pydantic, alert on validation failures, version your prompts.
• PII / data residency. If files contain personal data, check the provider’s data policy (do they train on inputs? where is data processed?). Use Bedrock / Azure OpenAI / Vertex if data must stay in a specific region.
• Non-text files. Scanned PDFs need OCR (Textract / Document AI / Form Recognizer) before the LLM. Plan for it; don’t send raw images to a text-only model.
• Backfills. When the prompt changes you’ll want to re-process old files. Keep the original file in object storage forever (or until policy says otherwise), and make file_id the natural key so re-runs upsert cleanly.
• Observability. Log structured events (file_id, latency, tokens, cost, status). Alert on DLQ depth and on the daily report failing to send.

Interview angle

Typical follow-ups:

1. “What if it grows to 10,000 files/day?” — Same architecture, more worker concurrency; the queue absorbs bursts. The LLM API rate limit becomes the bottleneck before your code does.
2. “How do you handle a file that the LLM consistently fails on?” — Dead-letter queue + manual review. Don’t let one bad file block the queue.
3. “Why not skip the queue and call the LLM directly from the upload handler?” — Synchronous = user waits 10s+, request timeouts, no retry on transient errors, no isolation from provider outages. The queue gives you all of that for free.
4. “How do you make the daily report exact, not ‘roughly the last 24h’?” — Use a processed_date column from the worker’s clock, run the report at a fixed time, query by date not interval.
5. “On-prem only — no cloud allowed. What changes?” — Swap S3 for MinIO, SQS for RabbitMQ/Redis, Lambda for a Celery worker, EventBridge for cron / Celery Beat, RDS for self-hosted Postgres. Architecture is identical; operational burden goes up.

Cross-links:

• LLM JSON schema enforcement: ../ai_ml/10_agents_orchestration/09_llm_json_validation.md
• Background workers (Celery vs asyncio): ../system_design/03_async_patterns/01_async_io_and_background_work.md
• Resilience patterns: ../system_design/02_resilience/