Why this matters
Modern backend systems are distributed. Event-driven architecture (EDA) helps services stay decoupled, resilient, and scalable. As a Backend Engineer, you will:
- Connect microservices using events instead of direct synchronous calls.
- Handle spikes gracefully with queues and backpressure.
- Achieve eventual consistency across services without global locks.
- Improve observability by tracing event flows.
Concept explained simply
An event is a fact that something happened, e.g., "OrderCreated". Services that care subscribe and react. The producer does not know who consumes the event; this keeps systems loosely coupled.
- Producer: emits events (e.g., Order Service).
- Broker: transports events (e.g., a topic or queue).
- Consumer: processes events (e.g., Inventory Service, Email Service).
- Event schema: shape of the payload; versioned to evolve safely.
- Delivery semantics: at-most-once, at-least-once, or effectively-once via idempotency.
- Ordering: per-key ordering is typical; global ordering is rare/expensive.
Mental model
Think "news bulletin board" not "phone call"
Instead of calling every service to tell them an order was created (phone calls), you post a note on a board (event). Anyone interested reads it when ready. If the board is busy, readers queue up; if a reader fails, they can resume where they left off.
Core patterns you will use
- Pub/Sub: many subscribers receive the same event.
- Queues: events load-balance across consumers for work distribution.
- Outbox pattern: reliably publish events written in the same DB transaction as state changes.
- Idempotent consumers: safely handle duplicates.
- Event versioning: add new fields as optional; keep old consumers working.
- Saga (process manager or choreography): long-running workflows coordinated by events.
Worked examples
Example 1: Order flow with inventory and email
Flow: Order Service emits OrderCreated. Inventory reserves stock, then emits InventoryReserved. Email Service sends confirmation when it sees OrderConfirmed.
{
"type": "OrderCreated",
"version": 1,
"id": "evt-01H...",
"occurred_at": "2026-01-20T10:00:00Z",
"data": {"order_id": "o-123", "user_id": "u-9", "items": [{"sku": "SKU1", "qty": 2}]},
"meta": {"trace_id": "tr-abc"}
}
Consumers:
- Inventory: checks stock, emits
InventoryReservedorInventoryFailed. - Email: waits for
OrderConfirmedto send email; ignoresOrderCreated.
Key ideas: loosely coupled services, eventual consistency, trace_id for observability.
Example 2: Payments with Outbox + idempotent handler
Payment Service writes PaymentCaptured to its DB and an outbox table within the same transaction. A relay publishes from outbox to the broker. The Order Service consumes and marks payment status. If the message is delivered twice, the consumer uses an idempotency key to ignore duplicates.
// Outbox row
{
"id": "out-77",
"event_type": "PaymentCaptured",
"aggregate_id": "order:o-123",
"payload": {"order_id": "o-123", "amount": 4999, "currency": "USD", "payment_id": "p-55"},
"status": "NEW"
}
Example 3: Search indexing and ordering
Catalog emits ProductUpdated events with a monotonically increasing version per product. Search Indexer partitions by product_id to preserve ordering per product. If events arrive out of order, the indexer discards stale versions.
{
"type": "ProductUpdated",
"version": 3,
"data": {"product_id": "p-1", "name": "Blue Tee", "price": 1999, "revision": 42}
}
Hands-on exercises
Do these now. They mirror the exercises section below and prepare you for the quick test.
Exercise 1 — Model key events for checkout
Design 3 events for an e-commerce checkout: OrderCreated, PaymentCaptured, OrderConfirmed. For each, write a minimal JSON payload and include an id, type, occurred_at, and a data object.
- [ ] Use consistent naming (snake_case or camelCase).
- [ ] Include
order_idon all events. - [ ] Add an optional
trace_idinmeta. - [ ] Keep fields additive; avoid breaking changes.
Exercise 2 — Make a consumer idempotent
Write pseudocode for a consumer handling PaymentCaptured that avoids double-charging or double-updating if the event is delivered more than once. Use a processed_events store keyed by event.id.
- [ ] Check if
event.idalready processed before applying changes. - [ ] Wrap state update and processed marker in a transaction.
- [ ] Make the handler retry-safe.
Exercise 3 — Sketch an Outbox flow
Draw or list the steps of the Outbox pattern for OrderConfirmed: table columns, how the relay reads and publishes, and how it marks rows as sent. Include failure/retry behavior.
- [ ] Single DB transaction persists domain change + outbox row.
- [ ] Relay is resilient, can resume by status/offset.
- [ ] Publishing is at-least-once; consumer is idempotent.
Common mistakes and self-check
- Confusing commands with events: Commands ask for action; events state facts. Self-check: event names past tense (e.g., OrderCreated), commands imperative (e.g., CreateOrder).
- Expecting exactly-once: Brokers typically provide at-least-once. Self-check: every handler idempotent? Duplicates safe?
- No schema versioning: Breaking consumers. Self-check: new fields optional? Is
versionpresent? - Global ordering assumption: Leads to bottlenecks. Self-check: partition by a key (e.g.,
order_id) and preserve per-key order. - Overusing events: Sometimes a synchronous call is better. Self-check: does the consumer truly need to react asynchronously?
- Missing observability: Hard to debug. Self-check: include
trace_id,correlation_id, consistent logs.
Who this is for
- Backend Engineers building microservices that must scale and stay decoupled.
- Engineers moving from monoliths to distributed systems.
- Platform engineers designing messaging standards.
Prerequisites
- Comfortable with REST basics and HTTP status codes.
- Know ACID transactions and basic SQL or NoSQL operations.
- Familiar with JSON data modeling.
Learning path
- Start here: event concepts, delivery semantics, idempotency.
- Next: message brokers and topics/partitions; saga patterns.
- Then: schema evolution, contract testing, and observability.
- Finally: performance tuning, backpressure, and capacity planning.
Practical projects
- Build a mini checkout flow with three services exchanging events in-memory (or via a simple queue) and add idempotency.
- Implement an Outbox table and a relay worker for reliable publish.
- Create a search indexer that consumes ProductUpdated events and ignores stale versions.
Next steps
- Harden your events: add
trace_id,version, and consistent naming. - Introduce retries with exponential backoff and dead-letter handling for poison messages.
- Add dashboards tracing event throughput, lag, and error rates.
Mini challenge
Design a payment refund flow using events. Define 3–4 events, show how idempotency is handled, and explain what happens if the refund fails midway. Keep each event minimal and versioned.
Take the Quick Test
Ready to check your understanding? Take the quick test below. Everyone can take it for free; if you are logged in, your progress will be saved automatically.