AI-Assisted Labeling

Using our label accelerator tools and AI-assisted labeling features

Track Forward

Sense makes it extremely easy to label image frames from video data. Track Forward uses AI to predict future video annotations, given object annotations in the current frame. The user provides annotations for the first frame using rectangles and polygons, and Track Forward predicts the annotations for those objects in the next frame. This allows for fast labeling of video datasets, since many object annotations can be generated automatically.

Track Forward is currently available for Rectangle and Polygon label types.

  1. When labeling a rectangle or polygon, enable Track Forward on that particular label by using the toggle You may need to click on the down arrow to select it. You'll know the toggle is enabled when the icon has a dark background.

  2. Select the label you want to apply and annotate the objects in the frame.

  3. Click "Submit". The next frame is loaded, and Track Forward will predict where the object(s) are in the frame and automatically annotate them.

Track Forward works best when:

  • Images are frames from a video segment. This feature is meant to track objects throughout frames in a video segment. It will not work for images that are completely different from each other.

  • Objects are not occluded or blocked from frame to frame.

  • Objects of the same type aren't overlapping and are spaced apart.

  • There isn't a lot of camera motion. Stationary (or nearly stationary) camera angles work best.

  • Objects are larger. As with most computer vision models, accuracy decreases as size of the object decreases.

Copy Forward

Labeling images with identical object placement from image to image (such as frames from a video) can be a tedious process, especially when objects appear in in multiple images at the same position. When objects are stationary from one image to another, it can save time to copy a label forward. While this feature is not assisted by AI, it accelerates labeling by saving the user time having to label stationary objects.

Copy Forward is available for Rectangle, Polygon, Point, and Feature Points label types.

  1. Enable Copy Forward on a label by using the toggle You may need to click on the down arrow to select it. You'll know the toggle is enabled when the icon has a dark background.

  2. Annotate the objects in the frame. Ensure that Copy Forward is toggled on for the labels that you want to copy.

  3. Click "Submit". The next frame is loaded, and Copy Forward will copy the label to the next image at the same position.


SmartPoly is a mode that lets you draw a rectangular bounding box around the object that is automatically transformed into a polygon.

To use SmartPoly, select the Polygon label type from the Labels bar.

  1. Click the icon on the label type to switch to SmartPoly mode. The icon will turn blue when SmartPoly is enabled.

  2. For best results, zoom in first so you can draw a tighter rectangle around the object.

  3. Select the location of your first corner and click and release, then drag the cross-hairs across the object so that the bounding box surrounds the object completely, and finally click and release at the opposite corner.

After a moment, the rectangular bounding box will turn into a polygon around the object.

SmartPoly works best when:

  • Object is against a contrasting background

  • Object is not overlapping with other objects

  • Bounding box is tight around the object