Image Tagging And Versioning

Why this matters

For an MLOps Engineer, image tags are the language of deployments. Clear, consistent tagging lets you: roll back a broken release instantly, reproduce a training run bit-for-bit, promote an image from staging to production without rebuilding, and audit exactly what ran last week. Poor tagging turns releases into guesswork and makes bugs hard to trace.

• Real tasks you will face: pinning training jobs to immutable images; promoting a model-serving image across environments; hotfixing and rolling back with confidence; matching images to Git commits and experiment runs.

Concept explained simply

An image tag is just a human-friendly label (like v1.4.2) pointing to an image. A digest (sha256:...) is the immutable fingerprint of the image content. Tags can move; digests never do. Use tags to communicate versions, and digests to guarantee exactness.

Mental model

• Tag = a sticky note that can be moved to a new box.
• Digest = the serial number etched into the box metal; it never changes.
• Use both: humans read tags, machines trust digests.

Practical tagging strategy (step-by-step)

1. Generate version data from source control and CI (e.g., semver + Git SHA + build time).
2. Build once, tag many: attach semver, channel, and Git SHA tags to the same image.
3. Push with all tags; record the resulting digest.
4. Pin deployments to the digest; keep tags for readability.
5. Promote by re-tagging the same digest to a new channel tag (no rebuilds).

# Build once, tag many
APP=my-app
VER=1.4.2
MAJOR=1
MINOR=1.4
SHA=$(git rev-parse --short HEAD 2>/dev/null || echo local)
REG=registry.example.com/team

docker build -t $REG/$APP:$VER -t $REG/$APP:$MINOR -t $REG/$APP:$MAJOR -t $REG/$APP:git-$SHA .

docker push $REG/$APP:$VER

docker push $REG/$APP:$MINOR

docker push $REG/$APP:$MAJOR

docker push $REG/$APP:git-$SHA

# Get immutable digest
DIGEST=$(docker inspect --format='{{index .RepoDigests 0}}' $REG/$APP:$VER)
echo $DIGEST  # e.g., registry.example.com/team/my-app@sha256:...

# Pin a deployment to the digest
# image: registry.example.com/team/my-app@sha256:...

# Promote to staging without rebuild
docker pull $DIGEST

docker tag $DIGEST $REG/$APP:staging

docker push $REG/$APP:staging

Worked examples

1) Stable API service release

Goal: Release 2.3.1 of a model-serving API, and promote to staging.

• Build: tags 2.3.1, 2.3, 2, git-a1b2c3.
• Push and record digest.
• Kubernetes: image uses the digest for the Deployment.
• Promotion: tag the same digest as staging and apply to the staging environment.

Result: Reproducible rollout; digest guarantees exact image content.

2) Reproducible training job

Goal: Tie a training run to a specific image so results are reproducible.

• Before run: log image digest to your experiment metadata (alongside code SHA and data version).
• Training pipeline: pulls by digest, not by moving tags.
• Re-run months later: pull the same digest to reproduce environment exactly.

3) Safe rollback from a bad hotfix

Goal: Version 3.1.0 hotfix breaks inference latency.

• You have the old digest for 3.0.5 from release notes.
• Rollback in minutes by deploying the previous digest.
• Optionally, retag the old digest as hotfix-rollback for clarity.

# Example with Docker Buildx
REG=registry.example.com/team
APP=trainer
VER=0.9.0

docker buildx build \
  --platform linux/amd64,linux/arm64 \
  -t $REG/$APP:$VER \
  -t $REG/$APP:0.9 \
  -t $REG/$APP:0 \
  --push .

# Pin deployments by digest of the manifest list
# kubernetes image: registry.example.com/team/trainer@sha256:...

Common mistakes and self-check

• Relying on latest in production. Fix: pin digests in manifests; keep latest only for local dev.
• Rebuilding for each environment. Fix: build once; promote by re-tagging the same digest (staging, prod).
• Missing link to source. Fix: include a Git SHA tag and label; store digest in release notes.
• Breaking consumers with moving minor tags. Fix: decide and document your tag policy; treat 1.4 as a convenience pointer, not a contract. Pin by digest where it matters.
• Ignoring multi-arch needs. Fix: publish manifest lists so tags resolve correctly on any node.

Exercises

Note: The quick test is available to everyone; only logged-in users get saved progress.

Exercise 1 (ex1): Build once, tag many

Goal: Build a local image and tag it with semver, minor, major, and git SHA; verify all tags point to the same digest.

1. Create a minimal Dockerfile that prints a message.
2. Build and tag as 1.2.3, 1.2, 1, and git-yoursha.
3. Use inspect to confirm they share the same RepoDigests value.

Exercise 2 (ex2): Promote by re-tagging a digest

Goal: Without rebuilding, promote an existing local image digest to a new tag staging and verify the digest matches.

1. Find the digest for your 1.2.3 image tag using docker inspect.
2. docker tag that digest to staging.
3. Compare digests of 1.2.3 and staging to ensure they match.

Who this is for

• MLOps Engineers owning training and serving pipelines.
• Data/ML Engineers standardizing reproducible environments.
• DevOps/SREs managing multi-environment Kubernetes deployments.

Prerequisites

• Comfort with Docker or compatible container tooling.
• Basic command-line skills.
• Familiarity with Git and semantic versioning is helpful.

Learning path

• Start: Image building basics and Dockerfile best practices.
• Next: Image tagging, digests, and semantic versioning.
• Then: CI/CD integration to build once and tag many.
• Later: Multi-arch images and manifest lists.
• Ongoing: Release notes that record tags and digests; audits and rollbacks.

Practical projects

• Release runner: Build a small CLI that reads version.json and outputs standardized tags (semver, minor, major, Git SHA).
• Promotion bot: Script that promotes a digest from dev to staging to prod by retagging and updating a Kubernetes image reference file.
• Repro trainer: Training pipeline that logs image digest, code SHA, and dataset version; includes a one-click re-run script pinned to those values.

Next steps

• Add labels (org.opencontainers.image.*) to embed metadata like source, revision, and build time.
• Introduce policy: enforce no latest in CI; require digest pins in production manifests.
• Automate release notes capturing tag sets and digests after each push.

Mini challenge

Your team needs a hotfix 2.0.1 for a serving image. Define the exact commands to: 1) build once and tag as 2.0.1, 2.0, 2, git-sha; 2) push; 3) promote the same digest to staging; 4) update deployment manifests to pin by digest; 5) record the digest in release notes. Keep it to five commands and one short notes line.