Event-Driven Microservices

Microservices that integrate through an asynchronous event backbone instead of synchronous request/response calls. Each service owns its data and reacts to events from others; no service blocks on another to do its job.

For the general event style see ../13_architecture_design/19_event_driven_architecture.md. For sync-vs-async transport choices see 02_inter_service_communication.md. For cross-service consistency see 04_data_consistency_patterns.md.

Why services reach for events

Synchronous call chains couple services in time — every callee must be up, fast, and reachable for the caller to succeed.

# Synchronous chain — fails and slows as a unit
Order → Payment → Inventory → Shipping
  if Inventory is down, the whole request fails
  total latency = sum of every hop
  Order is coupled to all three

# Event-driven — each service reacts on its own schedule
Order ──OrderPlaced──► (broker) ──► Payment
                                ──► Inventory
                                ──► Analytics
  a slow/down consumer doesn't fail the producer; events queue and drain later

Each service owns its data → eventual consistency

The defining constraint: one database per service, no shared tables, no cross-service joins, no distributed ACID transaction. State is synchronized by events, so the system is eventually consistent — there’s a window where services disagree, and the design must tolerate it (pending states, reconciliation, idempotent updates).

Event-carried state transfer kills the chatty read

To avoid calling another service on every request, a service keeps a local read replica of just the data it needs, updated from events.

# Shipping keeps its own copy of the customer address,
# fed by CustomerAddressChanged events — no call to Customer service at ship time.
def on_customer_address_changed(evt):
    local_addresses.upsert(evt.customer_id, evt.address)

def ship(order):
    addr = local_addresses.get(order.customer_id)   # local, fast, no remote dep
    courier.dispatch(order, addr)

This trades storage and eventual staleness for autonomy and latency — usually the right trade in microservices.

The dual-write problem is unavoidable here

Every service that mutates state and emits an event hits it: the DB commit and the publish can’t be one atomic action. Solve with the transactional outbox (write event + data in one local transaction, a relay publishes) or CDC reading the DB log (Debezium). See ../13_architecture_design/16_transactional_outbox.md.

def confirm_order(order):
    with db.transaction():          # one local transaction
        orders.update(order, status="CONFIRMED")
        outbox.insert(OrderConfirmed(order.id))   # not a second remote write
    # a separate relay reads outbox → publishes → marks sent

Coordinating multi-step flows

When one business operation spans services (place order → charge → reserve → ship), you can’t wrap it in a transaction. Use a saga: a sequence of local transactions with compensating actions to undo earlier steps when a later one fails.

• Choreography — services react to each other’s events; no coordinator. Good for short flows; logic gets scattered as steps grow.
• Orchestration — a coordinator (often Temporal / AWS Step Functions) drives steps and issues compensations. Preferred once the flow has several ordered steps.

Full treatment, including the “step N fails, roll back 1..N-1” rollback scenario: 04_data_consistency_patterns.md and ../13_architecture_design/15_event_driven_saga.md.

Schema is a cross-team contract

Events cross team boundaries, so the payload is a public API. Use a schema registry (Avro/Protobuf) with compatibility checks; add fields as optional; version the event (OrderPlaced.v2) for breaking changes. A careless payload change breaks every downstream team at once.

Observability is harder — design for it up front

A request that fans out across async consumers has no single stack trace. Thread a correlation/trace id through every event and propagate it into logs and spans, so one business operation can be reconstructed across services. See 03_distributed_tracing.md.

Choosing the backbone

See ../10_message_queues/kafka/01_what_is_kafka.md and ../10_message_queues/rabbitmq/01_what_is_rabbitmq.md.

Common pitfalls

• Distributed monolith — services that still call each other synchronously for every operation get microservice overhead with monolith coupling. Events are what break the coupling.
• Sync read in the hot path — calling another service on every request reintroduces temporal coupling. Replicate the data via events instead.
• No idempotency — at-least-once delivery means duplicates; non-idempotent consumers double-charge. Dedupe on event id.
• Naive dual write — DB write then publish without an outbox loses events on partial failure.
• Choreography sprawl — long event chains with logic spread across services become impossible to follow; orchestrate.
• Ignoring eventual consistency in the UX — show pending/processing states; don’t promise immediate global consistency you can’t deliver.

Interview angle

• “Why use events between microservices instead of REST?” — to remove temporal coupling: producers don’t block on consumers, failures don’t cascade, consumers scale and deploy independently, and new consumers attach without touching the producer.
• “Each service has its own DB — how do you keep data consistent?” — you don’t get distributed ACID; you get eventual consistency via events, with sagas + compensating transactions for multi-step operations and the outbox pattern for reliable publishing.
• “How does a service avoid calling another on every request?” — event-carried state transfer: keep a local read replica of the needed data, updated from the owning service’s events; read locally, accept slight staleness.
• “What’s the dual-write problem and how do you fix it in a service?” — a service can’t atomically commit its DB and publish an event; use a transactional outbox (or CDC) so the event is written in the same transaction and relayed afterward.
• “How do you debug a request that spans many services asynchronously?” — propagate a correlation/trace id through every event and into logs/spans (distributed tracing), so the whole flow can be reconstructed without a single call stack.
• “When is event-driven the wrong choice for services?” — when an operation needs an immediate synchronous answer, when strong consistency is required, or when the team can’t yet operate a broker and handle duplicates/ordering/observability.