Workflows vs Activities
The single most important Temporal distinction. Get this wrong and your workflows break in non-obvious ways during replay.
The split
| Workflow | Activity | |
|---|---|---|
| Role | orchestration — decide what runs and when | the actual work — call APIs, DBs, files |
| Must be deterministic | yes | no |
| Can do I/O | no (except via activities) | yes — anything |
| Survives crashes | yes (replayed from history) | no (retried by workflow) |
| Can sleep for days | yes (workflow.sleep(timedelta(days=30))) |
no (use heartbeats for long activities, but not days) |
| Can use random / time | only via workflow.random() / workflow.now() |
freely |
| Side effects | none allowed | this is where side effects live |
The mental model: workflow code is a recipe; activities are the cooking. Reading the recipe twice gives the same instructions. Cooking the same dish twice may give different results.
Why determinism
Temporal recovers workflow state by replaying the workflow code against the event history. Every decision the workflow made — which activity to call, what to pass it — must be reproducible from the history alone.
# WRONG — non-deterministic
@workflow.defn
class BadWorkflow:
@workflow.run
async def run(self):
if random.random() < 0.5: # different value on replay!
await workflow.execute_activity(path_a, ...)
else:
await workflow.execute_activity(path_b, ...)
On the original run, suppose random() returns 0.3 and path_a runs. The history records: “executed path_a, got result X”. On replay, random() returns 0.8 — the workflow now wants to call path_b, but the history says path_a was called. Non-determinism error: workflow execution panics.
# RIGHT — deterministic
@workflow.defn
class GoodWorkflow:
@workflow.run
async def run(self):
# workflow.random() uses a seeded PRNG; same value on every replay
if workflow.random().random() < 0.5:
await workflow.execute_activity(path_a, ...)
else:
await workflow.execute_activity(path_b, ...)
Things forbidden in workflow code
| Forbidden | Use instead |
|---|---|
time.time(), datetime.now() |
workflow.now() |
random.random(), uuid.uuid4() |
workflow.random(), workflow.uuid4() |
requests.get(...), any HTTP |
an activity |
| File I/O, DB queries | an activity |
| Threading, multiprocessing | structured concurrency via asyncio.gather |
os.environ reads at runtime |
pass values as workflow input |
asyncio.sleep |
workflow.sleep(timedelta(...)) |
| Mutable global state | workflow instance state only |
The Python SDK ships with a sandbox that intercepts many of these and raises at import or call time. You can mark trusted modules as passthrough.
Activities — everything you’d actually want to do
Activities are normal Python functions. They can fail, retry, time out, be cancelled.
from temporalio import activity
from temporalio.exceptions import ApplicationError
@activity.defn
async def charge_card(order_id: str, amount: int) -> str:
info = activity.info()
activity.logger.info(f"attempt {info.attempt} for {order_id}")
try:
return await stripe.charge(order_id, amount)
except StripeRateLimitError:
# Retryable — Temporal will retry per the policy
raise
except StripeInvalidCardError as e:
# Non-retryable — fail fast
raise ApplicationError("Invalid card", non_retryable=True) from e
Retry policy
Configured at the call site, not the activity:
from temporalio.common import RetryPolicy
await workflow.execute_activity(
charge_card,
args=[order_id, 1000],
start_to_close_timeout=timedelta(seconds=30),
retry_policy=RetryPolicy(
initial_interval=timedelta(seconds=1),
maximum_interval=timedelta(minutes=1),
backoff_coefficient=2.0,
maximum_attempts=5,
non_retryable_error_types=["InvalidCardError"],
),
)
Timeouts
Four timeouts; pick deliberately:
| Timeout | Meaning |
|---|---|
schedule_to_start |
how long the task can sit in queue before a worker picks it up |
start_to_close |
how long the activity has to run once started (the one you usually set) |
schedule_to_close |
end-to-end including retries |
heartbeat |
for long activities, how often the activity must call heartbeat() to prove it’s alive |
For long-running activities, heartbeat regularly so a worker crash mid-activity can be detected and retried:
@activity.defn
async def process_large_file(path: str):
for i, chunk in enumerate(read_chunks(path)):
process(chunk)
activity.heartbeat(i) # also persists progress for resume
Signals, queries, updates
Workflows aren’t just call-and-wait. They can be interacted with while running.
@workflow.defn
class SubscriptionWorkflow:
def __init__(self):
self._cancelled = False
self._plan = "basic"
@workflow.run
async def run(self, user_id: str):
while not self._cancelled:
await workflow.execute_activity(charge_monthly, user_id, self._plan, ...)
await workflow.sleep(timedelta(days=30))
@workflow.signal
def cancel(self): # fire-and-forget, no return value
self._cancelled = True
@workflow.signal
def change_plan(self, new_plan: str):
self._plan = new_plan
@workflow.query
def current_plan(self) -> str: # read-only, must not mutate
return self._plan
Then from any client:
handle = client.get_workflow_handle("subscription-123")
await handle.signal(SubscriptionWorkflow.change_plan, "premium")
plan = await handle.query(SubscriptionWorkflow.current_plan)
Signals mutate state. Queries read state synchronously without writing history. Updates (newer feature) are like signals but return a result.
continue_as_new
Workflows have a history size cap (~50k events by default). Long-running workflows (e.g., subscription that runs forever) hit this. Solution: at a clean boundary, start fresh:
@workflow.run
async def run(self, state: SubscriptionState):
for _ in range(100): # process 100 cycles, then restart
await charge_and_wait(state)
workflow.continue_as_new(state) # new execution, fresh history, same workflow ID
Common bugs
- “Workflow task failure: non-determinism” — you used a non-deterministic call in workflow code. Look for
datetime,random,requests, threading. - “Activity not registered” — worker started without the activity passed to its
activities=[...]list. - “Workflow worker not picking up tasks” — task queue name mismatch between starter and worker.
- Activity retries forever — you didn’t set
maximum_attemptsand the error isn’t marked non-retryable. - Signal arrives but doesn’t trigger logic — workflow is awaiting something blocking; use
workflow.wait_condition(lambda: self._something_changed)to react to signals.
Interview angle
- “Why split workflow code from activity code?” — workflows must be deterministic for replay; activities do all the side effects. The split makes durability mechanical instead of requiring the developer to checkpoint manually.
- “What happens if a worker dies mid-activity?” — the activity is retried per its retry policy on another worker (after the start_to_close timeout fires or the worker crashes detectably). If the activity was non-idempotent and partially completed, that’s on the application to design idempotently.
- “How do you handle a workflow that needs to wait 30 days?” —
await workflow.sleep(timedelta(days=30)). The workflow is unloaded from memory; Temporal wakes it up at the scheduled time. No worker resources are held during the wait. - “What’s
continue_as_newfor?” — to reset workflow history while preserving logical continuity, avoiding the ~50k event history cap on long-running workflows.