Instance Segmentor

Instance segmentors combine the advantages of semantic segmentors and object detectors. On the one hand, they are able to predict pixel-perfect masks like semantic segmentors achieving greater performance than object detectors. On the other hand, they are also able to distinguish between objects and counting them.

The most used architecture is Mask R-CNN, which conceptually speaking, does object detection first and then semantic segmentation inside the predicted bounding box.

The performance is usually evaluated using mAP.

If you're unsure if you should use object detection, instance, or semantic segmentation and start labeling your data: if you can count the object, then use instance segmentation. It's easy to use labels created for instance segmentation for object detection or semantic segmentation; the other way around is more tricky.

It is possible to combine semantic segmentation and instance segmentation in one image, then we talk about panoptic segmentation. The idea would be to label the road ahead as a semantic class and the cars as instances. This is not often used in practice, though (yet).

Further resources

Last updated on Jun 01, 2022

Get to production reliably.

Hasty is a unified agile ML platform for your entire Vision AI pipeline — with minimal integration effort for you.