Microservices And Integration

Why this skill matters for Backend Engineers

Microservices and integration let you ship features independently, scale specific components, and work across teams without blocking each other. You will define clear service boundaries, design stable contracts, choose the right communication patterns (HTTP/gRPC/events), and integrate safely with internal and third‑party systems. Mastering this unlocks reliable releases, better performance, and maintainable systems.

Who this is for

• Backend Engineers moving from monoliths to microservices.
• Engineers integrating multiple services or third‑party APIs.
• Developers aiming to improve reliability, evolvability, and team autonomy.

Prerequisites

• Comfortable with at least one backend language and REST fundamentals.
• Basic knowledge of queues or messaging is helpful (but not required).
• Familiar with Docker and environment variables for service configuration.

Learning path

1. Model service boundaries and contracts
   Outcome: You can define what a service owns and how it exposes APIs or events.
   How to practice

   • Pick a domain (e.g., Orders, Payments, Inventory). List each service's responsibilities and data ownership.
   • Draft a service contract (OpenAPI or gRPC proto) with inputs, outputs, and error codes.
2. Choose communication styles: HTTP vs gRPC vs events
   Outcome: You pick the right protocol for latency, throughput, and coupling.
   Quick guidance

   • HTTP+JSON: public APIs, browser/mobile, ad‑hoc integration.
   • gRPC: service‑to‑service, high throughput, typed contracts.
   • Events: decoupling, async workflows, fan‑out, audit.
3. Version and evolve contracts
   Outcome: You deploy changes without breaking consumers.
4. Discover services safely
   Outcome: You register, locate, and health‑check services automatically.
5. Handle distributed workflows
   Outcome: You coordinate multi‑service actions with sagas, idempotency, and compensation.
6. Integrate third‑party APIs and webhooks
   Outcome: You manage auth, retries, rate limits, and signature verification.

Worked examples

openapi: 3.0.0
info:
  title: Orders API
  version: 1.0.0
paths:
  /orders:
    post:
      summary: Create order
      requestBody:
        required: true
        content:
          application/json:
            schema:
              type: object
              required: [items, currency]
              properties:
                items:
                  type: array
                  items:
                    type: object
                    required: [sku, qty]
                    properties:
                      sku: { type: string }
                      qty: { type: integer, minimum: 1 }
                currency: { type: string }
                idempotencyKey: { type: string }
      responses:
        '201':
          description: Created
          headers:
            Location: { schema: { type: string } }
        '409': { description: Duplicate idempotency key }

syntax = "proto3";
package payments.v1;

message ChargeRequest {
  string order_id = 1;
  int64 amount_cents = 2;
  string currency = 3;
  string idempotency_key = 4; // for retries
}

message ChargeResponse {
  string charge_id = 1;
  string status = 2; // AUTHORIZED, DECLINED
}

service PaymentService {
  rpc Charge(ChargeRequest) returns (ChargeResponse);
}

# HTTP create order with idempotency
POST /orders
Idempotency-Key: 1f2d-...-9ab3
Content-Type: application/json

{
  "items": [{"sku": "SKU123", "qty": 2}],
  "currency": "USD",
  "idempotencyKey": "1f2d-...-9ab3"
}

# If retried with the same key, server returns the original result or 409

-- Outbox table
CREATE TABLE outbox (
  id UUID PRIMARY KEY,
  topic TEXT NOT NULL,
  payload JSONB NOT NULL,
  occurred_at TIMESTAMPTZ NOT NULL DEFAULT now(),
  status TEXT NOT NULL DEFAULT 'PENDING'
);

-- Application transaction (pseudocode)
BEGIN;
  INSERT INTO orders(id, status, total_cents) VALUES($1, 'CREATED', $2);
  INSERT INTO outbox(id, topic, payload)
    VALUES(gen_random_uuid(), 'order.created', json_build_object('orderId', $1, 'total', $2));
COMMIT;

// Relay loop (at-least-once)
while true:
  rows = select * from outbox where status='PENDING' limit 100 for update skip locked
  for r in rows:
    publish(r.topic, r.payload)
    update outbox set status='SENT' where id=r.id

// Order Service
on Command: PlaceOrder
  save Order(status='CREATING')
  emit Event: OrderCreated { orderId, total }

// Inventory Service
on Event: OrderCreated
  if reserve(stock): emit InventoryReserved { orderId }
  else emit InventoryRejected { orderId, reason }

// Payment Service
on Event: InventoryReserved
  if charge(): emit PaymentAuthorized { orderId }
  else emit PaymentDeclined { orderId, reason }

// Order Service reactions
on Event: PaymentAuthorized -> update status='CONFIRMED'; emit OrderConfirmed
on Event: InventoryRejected or PaymentDeclined ->
  emit CompensateReservation { orderId }
  update status='CANCELLED'

// Pseudocode
function isValidWebhook(requestBody, headers, secret):
  const signature = headers['X-Signature'] // e.g., hex hmac sha256
  const ts = parseInt(headers['X-Timestamp'])
  if (now() - ts > 5 * 60): return false // 5 min window
  const base = ts + '.' + requestBody
  const expected = HMAC_SHA256(base, secret)
  return constantTimeEqual(signature, expected)

Drills and exercises

• Define boundaries: write a one‑paragraph scope for Orders, Payments, Inventory. State what data each service owns.
• Write a minimal contract: one HTTP POST and one GET for a service. Include error codes and idempotency guidance.
• Add an event: define an order.created event payload with required and optional fields.
• Make a change: add a new optional field and prove it is backward‑compatible.
• Simulate failure: design compensating actions for a charge failure after inventory was reserved.
• Webhook drill: sketch signature verification and retry policy for 429/5xx responses.

Common mistakes and debugging tips

Practical projects

• Internal billing + invoicing: one service generates invoices from usage events; another handles payments; both publish lifecycle events.
• Notification hub: receives events and fans out via email/SMS/webhooks with retry and dead‑letter queues.
• Catalog + search: catalog publishes product updates; search service consumes and maintains a denormalized index.

Mini project: Orders + Payments with events and webhooks

Build two services and a mock third‑party payment provider that calls your webhook.

1. Define boundaries: Orders owns order state; Payments owns charge/refund.
2. Contracts:
   • Orders HTTP: POST /orders (idempotency), GET /orders/{id}
   • Payments gRPC: Charge(order_id, amount, currency, idempotency_key)
3. Eventing: Implement an outbox on Orders to emit order.created and order.confirmed.
4. Saga: Inventory is mocked; on order.created, reserve stock; on failure, emit compensation and cancel order.
5. Webhook: Mock provider POSTs /payments/webhook with HMAC signature. Verify signature and update payment status.
6. Reliability: Add retries with exponential backoff and jitter. Make POST operations idempotent.
7. Observability: Add request IDs in logs. Emit metrics for latency and error rate.

Subskills

• Service Boundaries And Contracts — Define ownership, APIs, and events for clear, stable interactions. Estimated: 45–90 min.
• Inter Service Communication Http Grpc — Choose and implement HTTP or gRPC with timeouts, retries, and idempotency. Estimated: 60–120 min.
• Event Driven Architecture Basics — Publish/subscribe events, outbox pattern, and at‑least‑once consumers. Estimated: 60–120 min.
• Schema And Contract Versioning — Backward/forward compatibility strategies and deprecation. Estimated: 45–90 min.
• Service Discovery Concepts — Client‑side/server‑side discovery, health checks, and config. Estimated: 45–90 min.
• Handling Distributed Transactions Basics — Sagas, compensation, and avoiding two‑phase commit. Estimated: 60–120 min.
• Integrating Third Party APIs — Auth, rate limits, retries, and resilience patterns. Estimated: 60–120 min.
• Webhooks And Callbacks — Secure receipt, replay handling, and verification. Estimated: 45–90 min.

Next steps

• Deepen reliability: add circuit breakers and bulkheads to your services.
• Add schema evolution tests to your CI using consumer‑driven contracts.
• Expand your mini project with a third service that consumes events and builds a read model.