Temporal — Common Interview Questions and Answers

1. What is Temporal and what problem does it solve?

Temporal is a durable execution platform — it runs orchestrating code (workflows) that survives crashes, restarts, and network failures without you writing state-recovery logic.

It solves the problem teams keep solving badly: multi-step business processes (order fulfillment, payments, onboarding) where partial failure between steps requires retries, compensating transactions, and persistent state. Without Temporal, teams reimplement this with DB tables tracking progress, ad-hoc retry logic, and brittle state machines.

2. What’s the difference between a workflow and an activity?

• Workflow — the orchestration code. Defines what runs and when. Must be deterministic (same inputs + history → same code path). No I/O, no datetime.now(), no random — only workflow.now() / workflow.random().
• Activity — the actual work. Calls APIs, DBs, files, anything. Can fail, retries are configured declaratively, non-deterministic is fine.

The split exists so Temporal can replay workflow code against the event history to reconstruct state after a crash. Activities are not replayed — their results are cached in history.

3. How does Temporal achieve durability?

Every workflow event (activity scheduled, activity completed, timer started, signal received) is written to an append-only event history persisted in Cassandra/PostgreSQL/MySQL.

When a worker crashes mid-workflow:

1. Another worker picks up the workflow from the task queue.
2. The worker replays the event history through the workflow code.
3. Activities that already completed return their cached results (no re-execution).
4. The workflow resumes from the first unfinished step.

This is why workflow code must be deterministic — the replay must produce the same sequence of decisions.

4. Why must workflow code be deterministic?

Temporal recovers workflow state by replaying the code, not by serializing in-memory state. If the code branches differently on replay (because random.random() returned a different value, or datetime.now() advanced), the replayed execution diverges from the original history — Temporal can’t reconcile that and the workflow task fails with a non-determinism error.

Forbidden in workflow code: wall-clock time, random, UUIDs, network calls, file I/O, threading, environment reads, mutable globals. Use the SDK’s deterministic equivalents (workflow.now(), workflow.random(), workflow.uuid4()) or move the operation into an activity.

5. What happens if a worker dies while running an activity?

The activity’s start_to_close timeout eventually fires (or the worker’s heartbeat stops). Temporal re-schedules the activity onto the task queue; another worker picks it up and runs it again per the retry policy.

If the activity is non-idempotent (e.g., “charge card”) and partially completed before the crash, that’s the application’s responsibility to handle — typically by making activities idempotent (use an idempotency key in the external API call) or by detecting “already done” state before acting.

6. How do you configure retries?

Per activity call, declaratively:

await workflow.execute_activity(
    charge_card,
    args=[order_id],
    start_to_close_timeout=timedelta(seconds=30),
    retry_policy=RetryPolicy(
        initial_interval=timedelta(seconds=1),
        maximum_interval=timedelta(minutes=5),
        backoff_coefficient=2.0,
        maximum_attempts=5,
        non_retryable_error_types=["InvalidCardError"],
    ),
)

Defaults: unlimited attempts, exponential backoff with 100ms initial interval, 2x coefficient. Mark errors non_retryable for fail-fast cases like validation errors.

7. What are the timeout types and when do you use each?

Set start_to_close always. Set heartbeat for anything > ~1 min so worker crashes are detected quickly.

8. How do you handle long-running workflows (days / weeks / months)?

Use workflow.sleep(timedelta(days=30)). The workflow is unloaded from worker memory during the sleep; Temporal wakes it up at the scheduled time on whatever worker is available. No resources are held during the wait.

For workflows that loop indefinitely (subscription billing), use workflow.continue_as_new(state) periodically — it starts a fresh workflow execution with reset history, preserving logical continuity. This avoids the ~50k event history cap.

9. What are signals and queries?

• Signal — fire-and-forget message to a running workflow. Mutates workflow state. Recorded in history.

  @workflow.signal
  def cancel(self):
      self._cancelled = True

  await handle.signal(MyWorkflow.cancel)

• Query — synchronous read of workflow state. Must not mutate. Not recorded in history.

  @workflow.query
  def status(self) -> str:
      return self._status

  s = await handle.query(MyWorkflow.status)

• Update (newer) — like signal but returns a result and can validate before accepting. Use when you need request/response semantics with state mutation.

10. How do you implement a saga / compensating transactions in Temporal?

Plain try/except. Compensations are normal activity calls in the except block:

@workflow.defn
class BookTripWorkflow:
    @workflow.run
    async def run(self, trip: Trip):
        flight_id = await workflow.execute_activity(book_flight, trip, ...)
        try:
            hotel_id = await workflow.execute_activity(book_hotel, trip, ...)
        except ActivityError:
            await workflow.execute_activity(cancel_flight, flight_id, ...)
            raise
        try:
            car_id = await workflow.execute_activity(book_car, trip, ...)
        except ActivityError:
            await workflow.execute_activity(cancel_hotel, hotel_id, ...)
            await workflow.execute_activity(cancel_flight, flight_id, ...)
            raise
        return BookingConfirmation(flight_id, hotel_id, car_id)

No state machine, no separate compensation orchestrator. Control flow is the orchestration.

11. How do you version workflows safely?

Old workflow executions continue running the code version they started with. New executions start with the new code. The problem: a workflow started yesterday is still running today against new code — if the code path changes, replay breaks.

Two main approaches:

• workflow.patched("v2-something") — branches behavior based on whether a “patch marker” exists in the history. Old workflows take the old path; new workflows take the new path.
• Task queue versioning / worker versioning — pin a workflow to a specific worker build ID.

For breaking changes, easiest is: drain old workflows, deploy new code, start new workflows on a new task queue.

12. What’s a task queue and how does routing work?

A task queue is a named queue workers poll. Workflows and activities are dispatched to task queues; workers poll specific queues and execute only what they’re registered for.

Common patterns:

• One task queue per workflow type (“orders”, “billing”).
• Separate queues for activities by required resources (e.g., “gpu-activities” polled only by GPU workers).
• Versioned queues for blue/green deploys.

Workers poll — Temporal Server doesn’t push. So workers need no inbound ports, scale horizontally trivially.

13. What are the failure modes?

• Activity failure — retried per policy; on exhaustion, exception propagates to workflow.
• Workflow task failure (the workflow code raised a non-application exception) — retried indefinitely by default; usually means non-determinism or a bug. Visible in Temporal UI as “Workflow Task Failed”.
• Worker crash — task is re-queued; another worker picks it up.
• Temporal Server outage — workflows are paused (no progress) but don’t lose state. Resume when server returns.
• History size cap (~50k events default) — workflow gets terminated. Mitigate with continue_as_new.
• Payload size cap (2MB default per event) — pass references (S3 URLs) for large data instead of inline payloads.

14. Temporal vs Celery — when do you pick each?

• Celery for fire-and-forget background tasks: send email, generate PDF, resize image. Simple, low operational overhead.
• Temporal for multi-step orchestrated processes where partial failure matters: order processing, payments, subscriptions, sagas.

Rule of thumb: if you’d otherwise write a state column in a DB tracking workflow progress, that’s where Temporal fits. If a task is “run this function once in the background,” Celery is fine.

Many teams run both — Celery for jobs, Temporal for workflows.

See 03_temporal_vs_celery.md for the full comparison.

15. What’s the operational cost of self-hosting Temporal?

Non-trivial. You need:

• The Temporal server cluster (4 services: frontend, history, matching, worker).
• A persistence backend — typically Cassandra (or PostgreSQL/MySQL for smaller scale).
• Elasticsearch for advanced visibility/search (optional but common).
• Monitoring for all of the above.

For teams without dedicated platform engineering, Temporal Cloud (managed) is often the right call. Pay per action; no cluster to operate.

16. How do you test workflows?

The SDK ships a test environment that runs workflows in-memory, optionally with time-skipping so a workflow.sleep(timedelta(days=30)) completes instantly.

async def test_order_workflow():
    async with await WorkflowEnvironment.start_time_skipping() as env:
        async with Worker(env.client, task_queue="test", workflows=[OrderWorkflow],
                          activities=[charge_card, reserve_inventory]):
            result = await env.client.execute_workflow(
                OrderWorkflow.run, "order-1",
                id="test-wf", task_queue="test",
            )
            assert result.status == "completed"

Activities can be mocked by registering mock implementations on the test worker. Workflow replay tests verify that current code can replay a saved history without non-determinism errors — critical before deploying changes.

17. Common pitfalls in production?

• Non-determinism creeping in — adding datetime.now() to a workflow that’s been deployed for months. Existing executions fail to replay. Always test replay against saved histories before deploying.
• Unbounded retries — forgetting maximum_attempts; an activity that always fails retries forever, filling history.
• Large payloads — passing big objects between activities; either hit the 2MB limit or bloat history. Pass references (S3 keys, DB IDs).
• Workflows that never continue_as_new — long-runners hit the history cap and get terminated.
• No separate task queue for heavy activities — one slow activity type starves quick ones on shared workers.
• Mixing workflow logic with activity-style side effects — caught by the sandbox in Python SDK; still happens.

18. How does Temporal compare to AWS Step Functions?

Both are durable workflow engines. Differences:

Pick Step Functions if you’re all-in on AWS and the workflow is straightforward. Pick Temporal for complex logic, multi-cloud, or long-running (> 1 year) workflows.

19. Can workflows call other workflows?

Yes — child workflows. Two patterns:

# Parent waits for child
result = await workflow.execute_child_workflow(ChildWorkflow.run, args)

# Fire and forget (parent doesn't wait)
await workflow.start_child_workflow(
    ChildWorkflow.run, args,
    parent_close_policy=ParentClosePolicy.ABANDON,  # child survives parent
)

Use child workflows for: composing reusable sub-workflows, parallel fan-out with independent retry policies, isolation of history size. Each child gets its own history.

20. What language SDKs does Temporal have?

Official SDKs: Go, Java, Python, TypeScript, .NET, PHP, Ruby. The Python SDK uses asyncio, runs the workflow sandbox via temporalio.worker.Worker, and ships with pytest-friendly testing utilities.

For Python: install with pip install temporalio. Run a local server via temporal server start-dev (single-binary dev mode, no Cassandra needed).

Quick reference: the mental model

Interview angle

• “What problem does Temporal solve?” - durable execution. Workflow state is persisted, so a multi-step process survives worker crashes, deploys and multi-day waits, and resumes exactly where it stopped. That’s the gap Celery leaves.
• “Workflow versus activity?” - workflow code must be deterministic because it’s replayed from history to rebuild state; activities hold all the non-deterministic work: I/O, randomness, time, external calls. Putting a network call in workflow code is the classic mistake.
• “Why must workflow code be deterministic?” - recovery replays the event history through your code. Anything that could produce a different result on replay - a timestamp, a random number, a map iteration order - corrupts the reconstructed state. Use the SDK’s provided time and random helpers.
• “Temporal or Celery?” - Celery for independent background tasks; Temporal for long-running multi-step processes needing retries, timeouts, compensation and visibility as first-class features rather than things you assemble.