Image annotation with boxes, polygons, and masks
Choose a visual annotation format based on what the model needs to locate or measure.
Match the annotation to the output
Bounding boxes work for locating objects when precise outlines are unnecessary. Polygons can capture irregular boundaries with fewer points than a detailed mask. Pixel masks are useful when the downstream task measures shape or area. A shelf inventory model may need a box per package, while a defect inspection model may need the outline of the damaged region.
Agree on boundary rules
Decide whether to label the visible part of an occluded object or infer its full extent. Define minimum sizes, handling for reflections, and how to mark objects cut off by the frame. Audit representative examples at the resolution reviewers actually use.
Work backward from the measurement
Imagine a model that finds scratches on a painted panel. If the inspection station only needs to send a worker to the right area, a box may be sufficient. If the decision depends on how much surface is damaged, a mask better supports an area measurement. For ribbons, cracks, or other thin shapes, inspect whether the target output preserves enough detail at the resolution the model will actually use.
Define how to mark regions with gaps
Set a rule for holes, disconnected regions, and objects that merge visually. For instance, if a handle forms an open loop, decide whether the empty center is excluded from the mask. State how the rule changes if the center is simply hidden. Labeling tools may represent disconnected pieces differently, so export a sample and inspect whether the format preserves those distinctions.
Compare formats on the same pilot batch
Select examples with easy boundaries, overlapping objects, and difficult lighting. Give reviewers the same images in the formats you are considering. Track time per image, missed instances, and the kinds of correction needed. Have the downstream team load exported annotations and score a few representative cases so the comparison includes integration effort as well as labeling time.
Check quality at the scale of the decision
Overlap scores can hide errors on small objects. Review misses and boundary errors by size and image condition. For area estimates, compare the marked area with a reference and inspect cases near the pass/fail boundary. For box detection, verify class, instance count, and coordinates after resizing. Keep a small set of these examples for future tool or dataset changes.