Image Preprocessing And Augmentation

Why this skill matters for Computer Vision Engineers

Image preprocessing and augmentation are core for training robust, fast, and reliable vision models. As a Computer Vision Engineer, you will clean and standardize inputs, simulate real-world conditions (lighting, blur, compression), balance classes, and improve generalization. This skill unlocks tasks like building reproducible data pipelines, reducing overfitting, handling aspect ratios without distorting objects, and safely augmenting labels for detection and segmentation.

Who this is for

• Engineers training classification, detection, or segmentation models.
• Data scientists moving from experimentation to production pipelines.
• Researchers needing controlled input distributions and robust evaluation.

Prerequisites

• Python basics and NumPy arrays.
• Familiarity with OpenCV or PIL and one DL framework (PyTorch or TensorFlow).
• Understanding of image classification; basics of detection/segmentation are helpful.

Learning path (practical roadmap)

1. Milestone 1: Reliable preprocessing
   • Convert color spaces consistently (BGR to RGB if using OpenCV).
   • Resize with the right interpolation; normalize with dataset mean/std.
   • Establish train/valid/test pipelines without leakage.
2. Milestone 2: Safe geometric transforms
   • Master crops, flips, rotations for classification.
   • Learn label-safe transforms for detection and segmentation.
3. Milestone 3: Photometric robustness
   • Apply color jitter, lighting changes, blur, noise, and compression.
   • Tune probabilities and ranges without destroying semantics.
4. Milestone 4: Advanced augmentation
   • Use Mixup/Cutmix for classification; monitor loss/accuracy behavior.
   • Adopt task-specific augmentations (mosaic, letterbox) when needed.
5. Milestone 5: Reusable pipelines
   • Wrap preprocessing and augmentation into reusable components.
   • Seed, log, and version your transforms for reproducibility.

Worked examples

# OpenCV read, RGB conversion, resize, and PyTorch tensor normalization
import cv2
import numpy as np
import torch

# Assume ImageNet-style stats for RGB [0-1]
mean = torch.tensor([0.485, 0.456, 0.406]).view(3,1,1)
std  = torch.tensor([0.229, 0.224, 0.225]).view(3,1,1)

img = cv2.imread('cat.jpg')            # BGR uint8 [0-255]
img = cv2.cvtColor(img, cv2.COLOR_BGR2RGB)
img = cv2.resize(img, (224, 224), interpolation=cv2.INTER_AREA)

x = torch.from_numpy(img).permute(2,0,1).float() / 255.0
x = (x - mean) / std
print(x.shape, x.min().item(), x.max().item())

# PyTorch: simple color jitter for classification
from torchvision import transforms

train_tfms = transforms.Compose([
    transforms.Resize(256),
    transforms.CenterCrop(224),
    transforms.ColorJitter(brightness=0.3, contrast=0.3, saturation=0.3, hue=0.05),
    transforms.ToTensor(),
    transforms.Normalize([0.485,0.456,0.406],[0.229,0.224,0.225])
])

# RandomResizedCrop + flip + small rotation
from torchvision import transforms

train_tfms = transforms.Compose([
    transforms.RandomResizedCrop(224, scale=(0.6, 1.0), ratio=(0.75, 1.33)),
    transforms.RandomHorizontalFlip(p=0.5),
    transforms.RandomRotation(degrees=10, expand=False, fill=0),
    transforms.ToTensor(),
    transforms.Normalize([0.485,0.456,0.406],[0.229,0.224,0.225])
])

# Simple Mixup implementation for a batch
import torch

def mixup_batch(x, y, alpha=0.4):
    bs = x.size(0)
    lam = np.random.beta(alpha, alpha)
    index = torch.randperm(bs)
    x_mixed = lam * x + (1 - lam) * x[index]
    y_mixed = lam * y + (1 - lam) * y[index]
    return x_mixed, y_mixed, lam, index

# Albumentations keeps bboxes/masks in sync
import albumentations as A

train_aug = A.Compose([
    A.LongestMaxSize(max_size=640),
    A.PadIfNeeded(min_height=640, min_width=640, border_mode=0, value=(114,114,114)),
    A.RandomBrightnessContrast(0.2, 0.2, p=0.5),
    A.GaussianBlur(blur_limit=(3,5), p=0.2),
    A.HorizontalFlip(p=0.5),
], bbox_params=A.BboxParams(format='pascal_voc', label_fields=['class_labels']),
   mask_params=A.MaskParams())

# Letterbox to target size while preserving aspect ratio
import cv2
import numpy as np

def letterbox(img, new_size=(640,640), color=(114,114,114)):
    h, w = img.shape[:2]
    r = min(new_size[0]/h, new_size[1]/w)
    nh, nw = int(round(h*r)), int(round(w*r))
    resized = cv2.resize(img, (nw, nh), interpolation=cv2.INTER_AREA)
    canvas = np.full((new_size[0], new_size[1], 3), color, dtype=img.dtype)
    top = (new_size[0] - nh) // 2
    left = (new_size[1] - nw) // 2
    canvas[top:top+nh, left:left+nw] = resized
    return canvas, r, (left, top)

Drills and exercises

• Compute dataset mean/std on your training set and update normalization.
• Create train and validation transforms; ensure validation has no random ops.
• Implement letterbox and verify bbox remapping on 10 random images.
• Tune color jitter to keep class identity intact (manual visual check).
• Add blur/noise/compression and confirm mAP/accuracy does not collapse.
• Try Mixup with alpha 0.2–0.8 and pick a stable value based on validation.

Common mistakes and debugging tips

• Using wrong channel order: OpenCV loads BGR; models expect RGB. Fix with color conversion before normalization.
• Normalizing uint8 directly: Divide by 255.0 before applying mean/std meant for 0–1 ranges.
• Stretching objects: Resizing without preserving aspect ratio can harm detection; use pad/letterbox.
• Leaking augmentation into validation: Keep validation deterministic; no random flips/jitter.
• Breaking labels: For detection/segmentation, always apply the same geometric transform to labels and clip bboxes.
• Over-augmentation: Excess rotations or heavy jitter can change class identity; start mild and monitor validation.

Mini project: Robust traffic sign classifier

Objective: Build a preprocessing and augmentation pipeline that improves robustness to lighting, blur, and compression for a small traffic sign classifier.

1. Preprocessing: Resize to 224, preserve aspect ratio for validation; normalize to ImageNet stats.
2. Augmentations (train only): moderate color jitter, horizontal flip where valid, small rotation, Gaussian blur, JPEG compression.
3. Experiment: add Mixup (alpha 0.4); track validation accuracy and confusion matrix.
4. Ablation: turn each augmentation off to see which contributes most.
5. Deliverables: code for reusable pipeline, charts of accuracy vs epoch, and 16-sample visualization grid.

Practical projects

• Defect detection under factory lighting: simulate flicker, shadows, motion blur, and lens smudges.
• Document scanner: handle skew, perspective transforms, grayscale normalization, and compression artifacts.
• Wildlife camera traps: heavy color/illumination shifts, random crops, and noise to mimic night IR cameras.

Next steps

• Complete the drills and mini project above.
• Review each subskill to close gaps.
• When ready, attempt the skill exam below. Anyone can take it; if you log in, your progress will be saved.

Skill exam

The exam tests your ability to design safe, effective preprocessing and augmentation for classification, detection, and segmentation. Your score is calculated immediately. Anyone can take it for free; logged-in users get saved progress.