Track Forward

Label objects and track them from frame to frame

Sense makes it extremely easy to label image frames from video data. Track Forward uses AI to predict future video annotations, given object annotations in the current frame. The user provides annotations for the first frame using rectangles and polygons, and Track Forward predicts the annotations for those objects in the next frame. This allows for fast labeling of video datasets since many object annotations can be generated automatically.

Track Forward is currently available for Rectangle and Polygon label types.

  1. When labeling a rectangle or polygon, enable Track Forward on that particular label by using the toggle You may need to click on the down arrow to select it. You'll know the toggle is enabled when the icon has a dark background.

  2. Select the label you want to apply and annotate the objects in the frame.

  3. Click "Submit". The next frame is loaded, and Track Forward will predict where the object(s) are in the frame and automatically annotate them.

Track Forward works best when:

  • Images are frames from a video segment. This feature is meant to track objects throughout frames in a video segment. It will not work for images that are completely different from each other.

  • Objects are not occluded or blocked from frame to frame.

  • Objects of the same type aren't overlapping and are spaced apart.

  • There isn't a lot of camera motion. Stationary (or nearly stationary) camera angles work best.

  • Objects are larger. As with most computer vision models, accuracy decreases as size of the object decreases.