Data Contracts And Schemas

Why this matters

APIs live and die by their contracts. A clear data contract and schema protect teams from accidental breaking changes, enable validation and code generation, and speed up integration across backend, mobile, and frontend. In day-to-day API engineering you will:

• Define request/response shapes and constraints that multiple services rely on.
• Validate payloads to catch bugs early and improve error messages.
• Evolve APIs safely without breaking existing clients.
• Generate models, validators, and clients from a single source of truth.

Concept explained simply

A data contract is an agreement about the shape and meaning of data exchanged between systems. A schema is the precise, machine-readable description of that contract (types, required fields, constraints, defaults).

Core concepts you will use

• Schema formats: JSON Schema (often via OpenAPI), Protocol Buffers, Avro, GraphQL SDL.
• Compatibility:
  • Backward compatible: old clients keep working with the new server.
  • Forward compatible: new clients work with old servers.
• Change types:
  • Additive (usually safe): add optional fields, new enum values (if clients tolerate unknowns).
  • Breaking: remove fields, make optional fields required, change types, reuse field numbers in Protobuf.
• Validation: enforce contract at boundaries (incoming requests and outgoing responses).
• Versioning strategies: URL prefix (v1, v2), header/media-type versioning, schema evolution with backward compatibility.
• Deprecation: mark fields/endpoints as deprecated, provide timelines and alternatives.

Worked examples

Example 1: JSON Schema for a simple resource

Goal: Define an Order response that is safe to extend later.

{
  "$schema": "https://json-schema.org/draft/2020-12/schema",
  "$id": "https://example.com/schemas/order.json",
  "type": "object",
  "additionalProperties": false,
  "required": ["id", "status", "items", "total"],
  "properties": {
    "id": {"type": "string", "format": "uuid"},
    "status": {"type": "string", "enum": ["pending", "paid", "shipped", "cancelled"]},
    "items": {
      "type": "array",
      "minItems": 1,
      "items": {
        "type": "object",
        "required": ["sku", "qty"],
        "additionalProperties": false,
        "properties": {
          "sku": {"type": "string", "minLength": 3},
          "qty": {"type": "integer", "minimum": 1}
        }
      }
    },
    "total": {"type": "number", "minimum": 0, "multipleOf": 0.01},
    "currency": {"type": "string", "pattern": "^[A-Z]{3}$"},
    "notes": {"type": "string", "maxLength": 500}
  }
}

Notes: currency is optional to allow adding a default later; enum values for status are explicit. Adding a new optional field is safe.

Example 2: Evolving a Protobuf message safely

Initial message:

message UserProfile {
  string id = 1;         // UUID
  string email = 2;      // required in business sense, optional in wire terms
  string name = 3;       // display name
}

We want to split name into first_name and last_name. Safe evolution:

message UserProfile {
  string id = 1;
  string email = 2;
  // name kept for backward compatibility; mark as deprecated in comments/docs
  string name = 3; // deprecated: use first_name + last_name
  string first_name = 4;
  string last_name = 5;
  // Reserve the old number if you ever remove a field in the future
  // reserved 3; // only after clients stop using 'name'
}

Rules: never reuse field numbers; add new fields with new numbers; deprecate before removal; reserve numbers when removing.

Example 3: OpenAPI request/response contract

Define a POST /payments that validates both request and response.

paths:
  /payments:
    post:
      requestBody:
        required: true
        content:
          application/json:
            schema:
              type: object
              required: [amount, currency]
              properties:
                amount: { type: number, minimum: 0, multipleOf: 0.01 }
                currency: { type: string, pattern: "^[A-Z]{3}$" }
                reference: { type: string, maxLength: 64 }
      responses:
        '201':
          description: Created
          content:
            application/json:
              schema:
                type: object
                required: [id, status]
                properties:
                  id: { type: string, format: uuid }
                  status: { type: string, enum: ["accepted", "declined", "pending"] }
                  message: { type: string }

Implementing a contract in 5 steps

1. Model the domain: list entities, fields, and constraints with the product team.
2. Write the schema: pick a format (OpenAPI/JSON Schema, Protobuf, GraphQL SDL).
3. Validate at boundaries: implement request and response validation in your API layer.
4. Automate: generate models and clients from the schema; run schema checks in CI.
5. Evolve safely: prefer additive changes; deprecate with timelines; version only when necessary.

Practice exercises

Do the exercises below, then use the checklist to self-verify. Full instructions are in the Exercises section of this page.

• Exercise 1: Author a Product JSON Schema with realistic constraints.
• Exercise 2: Evolve a Protobuf contract without breaking existing clients.

Common mistakes and how to spot them

• Making optional fields required later: this breaks old clients. Instead, keep them optional and validate server-side logic.
• Changing types (e.g., number to string for IDs): treat as breaking; version or introduce a new field with clear migration.
• Reusing Protobuf field numbers: this corrupts deserialization. Always add new numbers and reserve removed ones.
• Forgetting response validation: errors leak to clients. Validate responses in tests and at runtime where feasible.
• Overusing enums: they hinder flexibility. Use enums for stable sets; otherwise use strings with documented allowed values and a tolerance policy.

Practical projects

• Contract-first microservice: Design an OpenAPI for a Catalog service, generate server stubs and a TypeScript client, and implement two endpoints with validation.
• Schema evolution playbook: Take an existing Protobuf message and plan three additive changes with a deprecation path; write migration notes.
• Compatibility test suite: Create tests that fix a schema version and verify current code remains backward compatible.

Who this is for

• API Engineers and Backend Developers who integrate with multiple consumers.
• Developers moving from ad-hoc JSON to contract-first workflows.
• Teams adopting microservices or public APIs.

Prerequisites

• Basic HTTP and REST knowledge.
• Comfort with JSON/YAML. Some exposure to Protobuf or GraphQL is helpful but not required.
• Familiarity with unit/integration testing.

Learning path

• Start with Data Contracts and Schemas (this page).
• Then focus on Request/Response validation strategies and API error design.
• Move to Versioning and Deprecation practice across environments.
• Finally, adopt code generation and CI schema checks.

Next steps

• Complete the exercises and run the quick test.
• Add validation to one endpoint in your current project.
• Propose a deprecation policy template for your team.

Mini challenge

You need to add a new status "refunded" to an existing payment enum used by many clients. Describe a safe rollout plan in 3 steps. Hint: consider client tolerance for unknown values, deprecation notes, and monitoring.

Quick test

The quick test is available to everyone. Only logged-in users will have their progress saved.