Error Handling And Logging

Why this matters

Backends run in messy, unpredictable environments. Requests time out, services fail, data is malformed, and users still expect reliability. Good error handling and logging turn chaos into clear signals, reduce downtime, and speed up incident resolution.

• Map failures to predictable responses (e.g., 400 vs 500) so clients behave correctly.
• Record structured logs to diagnose issues quickly.
• Use timeouts, retries, and idempotency to make systems resilient.

Who this is for

• Junior to mid-level backend engineers who ship APIs or background jobs.
• Developers moving from scripts to production services.

Prerequisites

• Basic programming in one backend language (e.g., Python, Node.js, Go, Java).
• Familiarity with HTTP and JSON.
• Comfort running a local server and reading logs.

Learning path

1. Error taxonomy: Distinguish expected vs unexpected errors; map to HTTP status codes.
2. Logging fundamentals: Levels, structured JSON logs, correlation IDs, redaction.
3. Resilience patterns: Timeouts, retries with exponential backoff + jitter, idempotency keys, circuit breakers.
4. Operationalize: Sampling, retention, alerts, and on-call friendly messages.

Concept explained simply

An error is a deviation from the happy path. Some are expected (invalid input), others are unexpected (database down). Error handling decides what to do next: fail fast, retry safely, or degrade gracefully. Logging is how you capture what happened in a machine- and human-friendly way.

Mental model

Think of your service as an airport:

• Validation = security checkpoint (catch bad inputs early).
• Routing decisions = air traffic control (send errors to the right place, keep IDs to track planes).
• Logs = flight recorder (structured facts for post-incident analysis).
• Resilience = weather procedures (timeouts, retries, cancellations, alternate routes).

Core principles

• Error types:
  • Client/expected: validation errors, not found. Return 4xx; log at INFO or WARN with reason.
  • Server/unexpected: timeouts, null pointer, dependency down. Return 5xx; log at ERROR.
• HTTP mapping (examples): 400 bad params; 401 unauthenticated; 403 unauthorized; 404 not found; 409 conflict; 422 semantic validation; 429 rate limited; 500 internal; 502/503/504 dependency issues.
• Logging levels: DEBUG (dev detail), INFO (state change), WARN (recoverable oddity), ERROR (user impact), FATAL (process cannot continue).
• Structured logs: Emit JSON with stable keys: timestamp, level, message, service, correlation_id, error.code, error.stack (if any), http.status, user.id (if safe).
• Correlation IDs: Generate a request ID at the edge; pass to downstream calls; include in every log line.
• PII redaction: Never log secrets, passwords, tokens, full credit cards. Mask with *** or hashes.
• Timeouts: Bound every network call. A request without a timeout is a memory leak waiting to happen.
• Retries with backoff + jitter: Retry only idempotent operations. Use exponential backoff (e.g., 100ms, 200ms, 400ms) plus random jitter to avoid thundering herds.
• Idempotency: Ensure repeated requests produce the same result (e.g., idempotency keys for payments).
• Circuit breaker: If a dependency keeps failing, stop calling it briefly to let it recover and protect your threads.
• Observability quick wins: Add counters for error classes, latency histograms, and top error codes.

Worked examples

// Pseudocode
function POST /signup(request):
  correlation_id = request.headers["X-Request-ID"] or generate()
  log(INFO, {
    "message": "signup_received",
    "correlation_id": correlation_id
  })

  if not isValidEmail(request.body.email):
    log(WARN, {
      "message": "validation_failed",
      "field": "email",
      "code": "invalid_email",
      "correlation_id": correlation_id
    })
    return 400, {"error":"invalid_email","message":"Email format is invalid","correlation_id":correlation_id}

  // continue happy path...

function reserveInventory(orderId, items):
  idempotencyKey = hash(orderId)
  for attempt in 1..3:
    try:
      return callInventoryAPI({items, idempotencyKey}, timeout=750ms)
    except TimeoutError as e:
      wait = jitter(expBackoff(base=100ms, factor=2, attempt))  // 100ms, 200ms, 400ms ± jitter
      log(WARN, {
        "message": "inventory_retry",
        "attempt": attempt,
        "wait_ms": wait,
        "error": {"type":"timeout"}
      })
      sleep(wait)
  log(ERROR, {"message":"inventory_failed","order_id":orderId})
  throw DependencyError("inventory")

function processEmailJob(job):
  try:
    sendEmail(to=job.email, body=job.body, timeout=2s)
    log(INFO, {"message":"email_sent","to_domain":domain(job.email)})
  except PermanentError as e:
    log(ERROR, {"message":"email_permanent_fail","code":e.code,"to":"***@" + domain(job.email)})
    moveToDeadLetter(job)
  except TransientError as e:
    retryLater(job, delay=expBackoff(...))
    log(WARN, {"message":"email_transient_fail","retry_scheduled":true})

Before you ship — checklist

• Every external call has a timeout set.
• Retries only on idempotent paths and with backoff + jitter.
• All responses map to correct HTTP codes and machine-readable error bodies.
• Logs are structured JSON and include correlation_id.
• PII and secrets are redacted or excluded.
• Top 5 error classes are counted and visible.

Exercises

Note: Anyone can take the exercises and quick test. Only logged-in users see saved progress.

{
  "message":"charge_attempt",
  "email":"alex@example.com",
  "card":"4242424242424242",
  "authorization":"Bearer abcdef"
}

Common mistakes (and how to self-check)

• Logging errors without context: Add correlation_id, error.code, http.status. Self-check: Can you trace a single user request end-to-end using one ID?
• Over-retrying: Retrying non-idempotent actions causes duplicates. Self-check: If you replay the request 3 times, is the outcome unchanged?
• Missing timeouts: Hanging threads pile up. Self-check: Search code for external calls; confirm timeout parameter set everywhere.
• Logging secrets/PII: Compliance and security risk. Self-check: Scan for keys named password, token, authorization, card.
• Ambiguous HTTP codes: 500 for validation errors confuses clients. Self-check: For each error code, write a one-liner client action.

Practical projects

• Resilient User Service: Build a user signup/login API with structured logging, validation, rate limiting (429), and dependency timeouts.
• Job Worker with DLQ: A background processor that retries transient failures and moves permanent failures to a dead-letter queue.
• Log Enricher: A small library that injects correlation IDs and redacts PII, with unit tests.

Mini challenge

You receive an alert: error rate spiked from 0.5% to 4% after a deploy. You have 5 minutes.

• Which 3 log fields do you filter on first, and why?
• What single change would you roll back or feature-flag if logs show 503s from a dependency?
• How do you prevent a retry storm while mitigating user impact?

Next steps

• Complete the exercises below to practice error mapping, retries, and redaction.
• Take the quick test to confirm understanding. Aim for 80%+.
• Integrate a correlation ID middleware and a logging formatter in your current project.