API Development

Why API Development matters for Backend Engineers

APIs are how your backend delivers value to users, mobile apps, web frontends, and partner systems. Strong API development lets you ship features quickly, scale safely, and keep contracts stable as your product evolves.

• Turn business requirements into clean, predictable endpoints.
• Protect performance and data with validation, throttling, and pagination.
• Evolve without breaking clients via versioning and backward compatibility.
• Document clearly so others can integrate without hand-holding.

Who this is for

• Backend Engineers building services consumed by web, mobile, or partners.
• Full‑stack developers who want solid server APIs.
• Engineers moving from monoliths to service-oriented or microservice architectures.

Prerequisites

• Comfort with at least one backend language (e.g., JavaScript/TypeScript, Python, Go, Java).
• HTTP basics: methods, headers, status codes, JSON.
• Data modeling fundamentals and a database you can query.
• Basic command of a framework (Express, FastAPI, Spring, Gin, etc.).

Learning path

1. Design RESTful endpoints
   Define resources, nouns, and standard methods. Choose clear, consistent naming. Map status codes to outcomes.
2. Validate requests and responses
   Use schemas to reject bad input and guarantee outputs. Surface errors with structured payloads.
3. Pagination, filtering, sorting
   Return slices of data predictably. Avoid over-fetching and unbounded queries.
4. Versioning and compatibility
   Plan non-breaking changes and deprecation. Offer v1, v2 when needed.
5. Rate limiting and throttling
   Protect upstreams and fairness with simple token-bucket strategies.
6. Idempotency and retries
   Make unsafe operations safe to retry. Use idempotency keys and proper status codes.
7. Documentation with OpenAPI
   Publish a spec; auto-generate docs and clients.
8. File uploads and streaming
   Handle multipart uploads; stream large downloads efficiently.

Worked examples

1) RESTful endpoint with proper status codes (Node.js/Express)

const express = require('express');
const app = express();
app.use(express.json());

const products = new Map();
let idSeq = 1;

// Create
app.post('/v1/products', (req, res) => {
  const { name, price } = req.body;
  if (!name || typeof price !== 'number') {
    return res.status(400).json({ error: 'Invalid input' });
  }
  const id = idSeq++;
  const product = { id, name, price };
  products.set(id, product);
  res.status(201).json(product);
});

// Read
app.get('/v1/products/:id', (req, res) => {
  const id = Number(req.params.id);
  if (!products.has(id)) return res.status(404).json({ error: 'Not found' });
  res.json(products.get(id));
});

// Update (PUT replaces, PATCH partial)
app.patch('/v1/products/:id', (req, res) => {
  const id = Number(req.params.id);
  const p = products.get(id);
  if (!p) return res.status(404).json({ error: 'Not found' });
  const { name, price } = req.body;
  if (price !== undefined && typeof price !== 'number') {
    return res.status(400).json({ error: 'Invalid price' });
  }
  const updated = { ...p, ...(name ? { name } : {}), ...(price !== undefined ? { price } : {}) };
  products.set(id, updated);
  res.json(updated);
});

// Delete
app.delete('/v1/products/:id', (req, res) => {
  const id = Number(req.params.id);
  if (!products.has(id)) return res.status(404).json({ error: 'Not found' });
  products.delete(id);
  res.status(204).send();
});

app.listen(3000, () => console.log('API on :3000'));

Notes: 201 on create, 404 for missing, 204 on delete, and explicit validation.

2) Request/response validation (Python/FastAPI + Pydantic)

from fastapi import FastAPI, HTTPException
from pydantic import BaseModel, condecimal
from typing import List

app = FastAPI()

class ProductIn(BaseModel):
    name: str
    price: condecimal(gt=0)

class ProductOut(ProductIn):
    id: int

_db = {}
seq = 1

@app.post('/v1/products', response_model=ProductOut, status_code=201)
def create_product(p: ProductIn):
    global seq
    pid = seq
    seq += 1
    _db[pid] = { 'id': pid, **p.dict() }
    return _db[pid]

@app.get('/v1/products', response_model=List[ProductOut])
def list_products():
    return list(_db.values())

FastAPI enforces input/output schemas automatically and returns helpful errors.

3) Cursor pagination with filtering and sorting (Node.js/Express)

app.get('/v1/products', (req, res) => {
  const { limit = '10', cursor, q, sort = 'id' } = req.query;
  const lim = Math.min(parseInt(limit, 10) || 10, 100);
  let items = Array.from(products.values());
  if (q) items = items.filter(p => p.name.toLowerCase().includes(String(q).toLowerCase()));
  if (sort === 'price') items.sort((a,b) => a.price - b.price);
  else items.sort((a,b) => a.id - b.id);
  let start = 0;
  if (cursor) {
    const idx = items.findIndex(p => String(p.id) === String(cursor));
    start = idx >= 0 ? idx + 1 : 0;
  }
  const page = items.slice(start, start + lim);
  const nextCursor = page.length ? String(page[page.length - 1].id) : null;
  res.json({ items: page, nextCursor });
});

Clients request the next page using nextCursor, avoiding duplicates when data changes.

4) Versioning and backward compatibility (Go/Gin)

r := gin.Default()

v1 := r.Group("/v1")
{
  v1.GET("/products/:id", func(c *gin.Context) {
    // returns {id,name,price}
  })
}

v2 := r.Group("/v2")
{
  v2.GET("/products/:id", func(c *gin.Context) {
    // non-breaking: add field 'currency' defaulting to 'USD'
    // returns {id,name,price,currency}
  })
}

// Deprecation note via header
r.Use(func(c *gin.Context) {
  if strings.HasPrefix(c.Request.URL.Path, "/v1/") {
    c.Writer.Header().Set("Deprecation", "true")
    c.Writer.Header().Set("Sunset", "2026-12-31")
  }
  c.Next()
})

Expose v2 with additive changes; keep v1 operational with clear deprecation signals.

5) Idempotency for POST with retries (Node.js/Express)

const idempo = new Map(); // key -> response

app.post('/v1/orders', (req, res) => {
  const key = req.get('Idempotency-Key');
  if (!key) return res.status(400).json({ error: 'Idempotency-Key required' });

  if (idempo.has(key)) {
    const prev = idempo.get(key);
    return res.status(prev.status).json(prev.body);
  }

  // Simulate create
  const order = { id: Date.now(), items: req.body.items || [] };
  const body = { order };
  idempo.set(key, { status: 201, body });
  res.status(201).json(body);
});

Clients safely retry the same request with the same Idempotency-Key without creating duplicates.

6) File upload and streaming download (Node.js/Express)

const multer = require('multer');
const fs = require('fs');
const path = require('path');
const upload = multer({ dest: 'uploads/' });

app.post('/v1/files', upload.single('file'), (req, res) => {
  if (!req.file) return res.status(400).json({ error: 'No file' });
  res.status(201).json({ id: req.file.filename, original: req.file.originalname });
});

app.get('/v1/files/:id', (req, res) => {
  const filePath = path.join(__dirname, 'uploads', req.params.id);
  if (!fs.existsSync(filePath)) return res.status(404).json({ error: 'Not found' });
  res.setHeader('Content-Disposition', 'attachment');
  const stream = fs.createReadStream(filePath);
  stream.pipe(res);
});

Use streaming for large files to avoid loading everything into memory.

function rateLimit({ tokensPerInterval = 60, intervalMs = 60_000 }) {
  const buckets = new Map();
  return (req, res, next) => {
    const key = req.ip;
    const now = Date.now();
    const b = buckets.get(key) || { tokens: tokensPerInterval, ts: now };
    const refill = Math.floor((now - b.ts) / intervalMs) * tokensPerInterval;
    b.tokens = Math.min(tokensPerInterval, b.tokens + Math.max(0, refill));
    b.ts = now;
    if (b.tokens <= 0) {
      res.status(429).json({ error: 'Too Many Requests' });
    } else {
      b.tokens -= 1;
      buckets.set(key, b);
      next();
    }
  };
}

app.use(rateLimit({ tokensPerInterval: 100, intervalMs: 60_000 }));

Drills and exercises

• [ ] For an entity of your choice, write endpoint definitions for list/create/get/update/delete with expected status codes.
• [ ] Add JSON schema validation for one POST request and return a structured error body on failure.
• [ ] Implement cursor pagination returning items and nextCursor; verify stable ordering under concurrent inserts.
• [ ] Add rate limiting and log 429 responses with request identifiers.
• [ ] Make a POST endpoint idempotent; confirm two identical requests create exactly one record.
• [ ] Produce a minimal OpenAPI 3.0 spec for two endpoints and render it using a viewer.
• [ ] Upload a file and stream it back; verify memory usage stays steady for large files.

Common mistakes and debugging tips

• Overloading endpoints with verbs: Avoid paths like /createUser. Prefer nouns and HTTP methods: POST /users.
• Unbounded responses: Large lists without pagination can overload servers and clients. Always bound and sort.
• Vague errors: Return consistent error shapes with a code, message, and optional details. Log full context server-side.
• Breaking changes without versioning: Removing fields or changing types breaks clients. Additive changes are safest; otherwise introduce /v2.
• Ignoring idempotency: Network retries can create duplicates. Use Idempotency-Key or deterministic identifiers.
• Blocking on big files: Buffering entire files increases memory pressure. Stream uploads/downloads.
• Insufficient timeouts: Clients and servers need sensible timeouts and retry policies to avoid hangs.

Mini project: Product Catalog API

Build a small API for products and categories with the following requirements:

• Endpoints: list/create/get/update/delete for /products and /categories.
• Relations: a product can belong to many categories; include filtering by category.
• Validation: enforce name (string, required) and price (positive number).
• Pagination: cursor-based for listing; include nextCursor.
• Versioning: v1 returns products with id, name, price; v2 adds currency (default USD).
• Idempotency: POST /orders with Idempotency-Key to simulate purchases.
• Rate limiting: 100 requests/min per IP.
• Docs: OpenAPI spec covering at least 6 endpoints.
• Files: allow uploading a product image and streaming it back.

Subskills

• Designing RESTful Endpoints
• Request Response Validation
• Pagination Filtering Sorting
• Versioning And Backward Compatibility
• Rate Limiting And Throttling Basics
• Idempotency And Retries
• API Documentation OpenAPI Basics
• Handling File Uploads And Streaming Basics

Next steps

• Complete the drills, then build the mini project end-to-end.
• Take the skill exam below to confirm mastery. Anyone can take it; sign in to save progress.
• Move to adjacent skills like authentication/authorization, caching, and observability.