Why this skill matters for Computer Vision Engineers
Computer Vision Foundations give you the tools to turn raw pixels into reliable signals for models and products. You will understand how images are represented, how cameras capture them, how to transform them geometrically, how to extract features, and when to choose classical methods versus deep learning. Mastering these basics unlocks tasks like robust preprocessing, fast prototypes, reproducible experiments, and shipping models that work beyond the lab.
What you'll be able to do
- Load, inspect, and convert images across color spaces (RGB, BGR, HSV, Grayscale).
- Apply geometric transforms (resize, crop, rotate, affine/perspective warp) safely.
- Extract edges and corners, and use convolutional filters effectively.
- Build minimal deep learning pipelines for vision (data, model, training loop).
- Spot dataset biases and measure performance beyond overall accuracy.
- Set up reproducible experiments with fixed seeds and tracked configurations.
Who this is for
- Aspiring Computer Vision Engineers starting from Python/ML basics.
- Data Scientists moving from tabular NLP/audio into image tasks.
- Engineers building vision features into products (quality, safety, logistics, retail, robotics).
Prerequisites
- Python basics: functions, packages, virtual environments.
- NumPy arrays and basic linear algebra (vectors, matrices, dot product).
- Optional but helpful: Familiarity with PyTorch or similar deep learning library.
Learning path
Milestone 1 — Images and color spaces (1–2 hours)
- Load images, inspect shapes and dtypes, convert between BGR/RGB/HSV/GRAY.
- Plot channel histograms; try simple thresholding in different spaces.
Milestone 2 — Geometric transforms (1–2 hours)
- Resize with correct interpolation (area vs nearest vs linear).
- Rotate, crop, affine vs perspective transforms; understand when each is safe.
Milestone 3 — Convolution, edges, corners (2 hours)
- Apply Sobel, Laplacian, Canny; detect corners (Harris/FAST).
- Understand kernels, stride, padding; visualize filter effects.
Milestone 4 — Camera basics (1 hour)
- Pinhole model, intrinsics (K), distortion basics, FOV vs focal length.
Milestone 5 — Deep learning basics for vision (2–3 hours)
- Dataset/DataLoader, simple CNN, training loop, basic augmentations.
Milestone 6 — Dataset bias and reproducibility (1 hour)
- Compute per-class metrics, confusion matrix; fix seeds; log configs and versions.
Worked examples
Example 1 — Color spaces and thresholding
import cv2
import numpy as np
img_bgr = cv2.imread('image.jpg') # shape: (H, W, 3), BGR order
img_rgb = cv2.cvtColor(img_bgr, cv2.COLOR_BGR2RGB)
img_hsv = cv2.cvtColor(img_bgr, cv2.COLOR_BGR2HSV)
img_gray = cv2.cvtColor(img_bgr, cv2.COLOR_BGR2GRAY)
# Threshold in HSV to isolate reds (hue ~ 0 or ~180)
lower_red1 = np.array([0, 100, 80])
upper_red1 = np.array([10, 255, 255])
lower_red2 = np.array([170, 100, 80])
upper_red2 = np.array([180, 255, 255])
mask1 = cv2.inRange(img_hsv, lower_red1, upper_red1)
mask2 = cv2.inRange(img_hsv, lower_red2, upper_red2)
mask = cv2.bitwise_or(mask1, mask2)
# Compare with grayscale threshold
_, mask_gray = cv2.threshold(img_gray, 120, 255, cv2.THRESH_BINARY)
# Observations: HSV hue thresholding is often more robust to lighting than grayscale.
Example 2 — Safe geometric transforms
import cv2
img = cv2.imread('image.jpg')
# Resize: use INTER_AREA for downscaling, INTER_LINEAR for upscaling
small = cv2.resize(img, (0,0), fx=0.5, fy=0.5, interpolation=cv2.INTER_AREA)
large = cv2.resize(img, (img.shape[1]*2, img.shape[0]*2), interpolation=cv2.INTER_LINEAR)
# Rotate around center by 30 degrees
h, w = img.shape[:2]
M = cv2.getRotationMatrix2D((w/2, h/2), 30, 1.0)
rotated = cv2.warpAffine(img, M, (w, h), flags=cv2.INTER_LINEAR, borderMode=cv2.BORDER_REFLECT)
# Perspective warp: map 4 points (when camera tilt causes foreshortening)
src = np.float32([[10,10],[w-10,20],[15,h-20],[w-25,h-10]])
dst = np.float32([[0,0],[w,0],[0,h],[w,h]])
H = cv2.getPerspectiveTransform(src, dst)
rectified = cv2.warpPerspective(img, H, (w, h))
Example 3 — Convolution filters, edges, and corners
import cv2
import numpy as np
img = cv2.imread('image.jpg', cv2.IMREAD_GRAYSCALE)
# Sharpening kernel
kernel_sharp = np.array([[0,-1,0],[-1,5,-1],[0,-1,0]], dtype=np.float32)
sharpened = cv2.filter2D(img, -1, kernel_sharp)
# Edges
edges = cv2.Canny(img, 100, 200)
# Corners (Harris)
img_float = np.float32(img)
H = cv2.cornerHarris(img_float, blockSize=2, ksize=3, k=0.04)
H_norm = cv2.normalize(H, None, 0, 255, cv2.NORM_MINMAX)
# Threshold corners for visualization
corners = (H_norm > 125).astype(np.uint8) * 255
Example 4 — Camera intrinsics and undistortion
import cv2
import numpy as np
img = cv2.imread('distorted.jpg')
# Example intrinsics and distortion (normally obtained via calibration)
K = np.array([[800, 0, 640],[0, 800, 360],[0, 0, 1]], dtype=np.float32)
dist = np.array([-0.2, 0.05, 0.0, 0.0], dtype=np.float32) # k1, k2, p1, p2 (simple example)
h, w = img.shape[:2]
newK, roi = cv2.getOptimalNewCameraMatrix(K, dist, (w, h), alpha=0.0)
undistorted = cv2.undistort(img, K, dist, None, newK)
# Note: Use a real checkerboard calibration to estimate K and distortion accurately.
Example 5 — Minimal CNN training loop (PyTorch)
import torch, torch.nn as nn, torch.optim as optim
from torch.utils.data import DataLoader, TensorDataset
import numpy as np
# Fake tiny dataset: 32x32 grayscale, 2 classes
np.random.seed(0)
X = np.random.rand(200, 1, 32, 32).astype('float32')
y = np.random.randint(0, 2, size=(200,)).astype('int64')
dataset = TensorDataset(torch.tensor(X), torch.tensor(y))
loader = DataLoader(dataset, batch_size=32, shuffle=True)
class SmallCNN(nn.Module):
def __init__(self):
super().__init__()
self.net = nn.Sequential(
nn.Conv2d(1, 8, 3, padding=1), nn.ReLU(),
nn.MaxPool2d(2),
nn.Conv2d(8, 16, 3, padding=1), nn.ReLU(),
nn.AdaptiveAvgPool2d((1,1))
)
self.fc = nn.Linear(16, 2)
def forward(self, x):
x = self.net(x)
x = x.view(x.size(0), -1)
return self.fc(x)
model = SmallCNN()
opt = optim.Adam(model.parameters(), lr=1e-3)
loss_fn = nn.CrossEntropyLoss()
def train_epoch():
model.train()
total, correct, loss_sum = 0, 0, 0.0
for xb, yb in loader:
opt.zero_grad()
logits = model(xb)
loss = loss_fn(logits, yb)
loss.backward()
opt.step()
loss_sum += float(loss)
preds = logits.argmax(dim=1)
correct += int((preds == yb).sum())
total += yb.size(0)
return loss_sum/len(loader), correct/total
for epoch in range(3):
loss, acc = train_epoch()
print(f"epoch {epoch+1}: loss={loss:.3f}, acc={acc:.2f}")
Example 6 — Check dataset bias and per-class metrics
import numpy as np
# Ground truth and predictions for a 3-class problem
y_true = np.array([0,0,1,1,1,2,2,2,2])
y_pred = np.array([0,1,1,1,0,2,2,1,2])
classes = np.unique(y_true)
cm = np.zeros((len(classes), len(classes)), dtype=int)
for t, p in zip(y_true, y_pred):
cm[t, p] += 1
print("Confusion matrix:\n", cm)
# Per-class recall = TP / (TP + FN)
recall = []
for c in classes:
tp = cm[c, c]
fn = cm[c, :].sum() - tp
r = tp / (tp + fn) if (tp + fn) else 0.0
recall.append(r)
print("Per-class recall:", recall)
# If one class has notably worse recall, investigate sampling, augmentations, and labeling.
Drills and exercises
- Load three images with different lighting. Segment a colored object using HSV; compare with grayscale thresholding.
- Rotate an image by 10°, 45°, 90° and inspect interpolation artifacts. Try BORDER_REFLECT vs BORDER_CONSTANT.
- Apply Sobel X/Y and visualize gradient magnitude. Vary kernel size and note edge thickness.
- Detect corners with Harris and FAST; compare counts and locations.
- Create a perspective warp to deskew a photographed document.
- Train the SmallCNN on random data, then on a tiny real dataset if available. Observe overfitting signs.
- Compute a confusion matrix on your validation set and report per-class recall.
- Fix seeds in Python, NumPy, and PyTorch; rerun your training twice and compare results.
Common mistakes and debugging tips
Mistake: Using RGB with OpenCV functions that expect BGR
Tip: OpenCV loads as BGR by default. Convert with cv2.cvtColor(img, cv2.COLOR_BGR2RGB) only when visualizing or mixing libraries.
Mistake: Resizing with wrong interpolation
Tip: Use INTER_AREA for downscaling (less aliasing), INTER_LINEAR/INTER_CUBIC for upscaling.
Mistake: Overreliance on accuracy
Tip: Always check per-class precision/recall and confusion matrix to uncover dataset bias.
Mistake: Unstable experiments
Tip: Fix random seeds (Python, NumPy, framework), control data order, and log versions/configs. Save checkpoints with metadata.
Mistake: Perspective warp used when affine would suffice
Tip: If lines are parallel in the scene, an affine transform is simpler and more stable. Use perspective only when needed (projective effects).
Mini project — Color-robust object localization pipeline
Goal: Build a classical pipeline to localize a colored object across varied lighting and viewpoints, then benchmark a tiny CNN classifier on cropped patches.
- Step 1: Collect 50–100 images of the object in different lighting and backgrounds.
- Step 2: Preprocess with HSV thresholding + morphology (open/close) to create a mask.
- Step 3: Find contours and compute bounding boxes; filter by area and aspect ratio.
- Step 4: Crop candidate patches. Split into train/val for a tiny classifier (object vs not-object).
- Step 5: Train the tiny CNN (like SmallCNN with 2 outputs). Evaluate per-class recall.
- Step 6: Make the workflow reproducible: fix seeds, save config (thresholds, kernel sizes), and log results.
Success criteria checklist
- ≥ 80% recall on object class on validation set.
- Consistent results across two reproducible runs (similar accuracy).
- Readable code with comments and saved configuration parameters.
Practical projects you can build next
- Document deskewer: detect corners of a document and rectify with perspective warp.
- Edge-based quality check: use Canny and contour features to flag deformed parts.
- Simple pose proxy: detect chessboard corners and estimate homography for planar tracking.
Subskills
- Image Representation and Color Spaces — Understand pixel layouts, dtypes, and when to use RGB/HSV/GRAY for robust preprocessing.
- Geometric Transformations Basics — Resize, crop, rotate, affine and perspective transformations with the right interpolation.
- Camera and Optics Basics — Pinhole model, intrinsics, distortion, and field of view vs focal length.
- Convolution and Feature Extraction — Kernels, stride, padding; edge/sharpen filters and feature maps.
- Classical Vision Concepts: Edges and Corners — Sobel, Canny, Harris/FAST and their use cases.
- Deep Learning for Vision Basics — Data pipeline, small CNNs, training loop, and basic augmentations.
- Dataset Bias Awareness — Class imbalance, per-class metrics, and fair evaluation practices.
- Reproducible Vision Workflows — Seeding, versioning, configuration logging, and deterministic runs when possible.
Next steps
- Finish the drills and the mini project above.
- Take the skill exam below to validate your understanding.
- Then continue to more advanced topics: image augmentation strategies, feature matching, object detection, and training production-ready models.