Bounding Boxes And Polygons

Who this is for

People preparing for Computer Vision Engineer roles or anyone building datasets for object detection and instance segmentation. You will learn to annotate consistently with bounding boxes and polygons.

Prerequisites

• Basic image concepts: pixels, width/height, coordinate origin (top-left).
• Comfort with annotation tools (any). Keyboard shortcuts help.
• Ability to follow written guidelines strictly and repetitively.

Why this matters

High-quality bounding boxes and polygons directly affect model accuracy. In real projects you will:

• Prepare training data for object detection (boxes) and instance segmentation (polygons).
• Define and enforce annotation rules for teams.
• Run quality control (QC) using IoU thresholds and spot checks.
• Balance annotation speed vs. precision based on project goals.

Concept explained simply

• Bounding box: The tightest axis-aligned rectangle that contains the object. It is fast to draw and used for detection models.
• Rotated box: A rectangle that can rotate. It fits elongated, angled objects better than axis-aligned boxes. Only use if your tooling and spec require it.
• Polygon (instance): A sequence of points around the object boundary, typically capturing the visible part (modal), unless the spec requests amodal (full extent). More precise, slower to create.

Key terms:

• Tightness: How closely your annotation hugs the object. Boxes should be tight but not crop the object. Polygons should adhere to edges without zig-zag noise.
• IoU (Intersection over Union): Overlap metric used in QC: IoU = area(intersection) / area(union). Often 0.5+ is acceptable for boxes; segmentation benchmarks may require higher.
• Occlusion vs. truncation: Occlusion = blocked by another object. Truncation = cut off by image boundary. Mark attributes if your spec requires.

Mental model

Think of a box as a rubber band stretched to cover the furthest visible extremes of the object along x/y. Think of a polygon as connect-the-dots along the object boundary, placing points only where direction changes. Fewer, smarter points produce cleaner masks.

Standards you should set before annotating

• Visible vs. amodal: Default to visible area unless the project states otherwise.
• Minimum object size: Decide a pixel threshold to annotate or skip tiny objects.
• Class list and attributes: Clear names, no overlaps between classes, and attribute tags for occluded/truncated if needed.
• Allowed shapes: Box only, box+rotated box, or box+polygon. Keep it consistent throughout the dataset.
• Quality rules: IoU target for box agreement (e.g., 0.5 or 0.6), polygon vertex spacing (denser on curves, sparse on straight lines).

How to draw accurate annotations (step cards)

1. Frame the task. Confirm class, shape type (box/polygon), and visible vs. amodal rule.
2. Place rough shape. For boxes, drag corner-to-corner covering the object. For polygons, drop anchor points around major corners first.
3. Refine. For boxes, nudge edges until they are tight without clipping. For polygons, add points only where the contour changes direction. Avoid unnecessary vertices.
4. Verify. Zoom in near edges. Ensure no large background gaps inside the shape and minimal foreground outside.

Worked examples

Quality control: self-check

• Consistency check: Compare 3–5 similar objects; are tightness and inclusion rules the same?
• Edge check: Zoom to edges; is there a clear, small background margin but no clipping?
• IoU sanity: If re-drawing a box, does your redo overlap the original with IoU ≥ target (e.g., 0.5)?
• Polygon health: No self-crossing, minimal redundant points, sharper corners where needed.

Common mistakes and how to self-correct

• Boxes too loose: Refit by nudging edges inward while keeping the full object inside.
• Clipping object: If any part is cut off, expand slightly; better slightly loose than clipped.
• Over-dense polygons: Remove needless points on straight segments; keep vertices at curvature changes.
• Including occluders: Exclude the blocking object unless the spec states amodal.
• Inconsistent class boundaries: Re-read class definitions; if two classes are ambiguous, add a rule and stick to it.

Practical projects

• Mini dataset: Capture 50 everyday images. Annotate 3 classes (e.g., bottle, phone, cup). Use boxes for all, polygons for 20 images. Write 8–10 rules you followed.
• QC drill: Re-annotate 10 images a day later. Compute box IoU by measuring coordinates in your tool. Aim for consistency ≥ 0.5 IoU.
• Attribute practice: Add occluded/truncated flags where appropriate and review disagreements.

Exercises you can do now

These mirror the tasks in the Exercises section below. After finishing, compare with the provided solutions. No special software required—use any annotation tool or sketch on paper to simulate bounding decisions.

Exercise 1: Tight bounding boxes with occlusion

Pick a photo with at least 5 objects; ensure 1–2 are partially hidden. Annotate boxes:

• Follow visible rule (default).
• Keep boxes tight without clipping.
• Mark occluded/truncated status if applicable.

Exercise 2: Clean polygons on irregular shapes

Pick 3 irregular objects (leaf, shoe, mug with handle). Draw polygons:

• Sparse points on straights; denser on curves.
• No self-intersections.
• Include only visible area; exclude shadows and reflections.

Mini challenge

Two annotators drew boxes for the same object. Box A area = 1200 px, Box B area = 1000 px, intersection area = 700 px. Is IoU ≥ 0.5?

Learning path

• Start: Bounding boxes on simple scenes → add occlusion and truncation cases.
• Advance: Polygons for irregular objects → refine vertex placement.
• QC: Practice IoU checks and consistency audits.
• Format: Learn common export schemas (names and fields) and how to validate them.

Next steps

• Create a small class guide with positive/negative examples for your 3 classes.
• Define measurable QC rules (e.g., IoU targets, polygon vertex density).
• Move to multi-annotator agreement: sample 10 images, compare, and resolve rules.