Grouping Annotations via Tracks

📘

Beta Feature

Viewing Tracks in Nucleus is available on an on-demand basis. Please contact Scale to enable this feature for your team.

In cases that multiple annotations with the same label are defined on a dataset item or a frame within a scene, it is useful to distinguish which annotations across items point to the same real-world object.

An example would be detecting the same red car, which has a ground truth label "car", across multiple frames of a scene. The "red car", in this instance, is a Track, and it forms a one-to-many relationship with its Ground Truth Annotations.

Creating Tracks

Tracks must be defined explicitly when uploading annotations. To do so, supply the track_reference_id to any type of annotation or model prediction:

from nucleus import BoxAnnotation

box_gt_w_track = BoxAnnotation(
    label="car",
    x=4,
    y=6,
    width=12,
    height=18,
    reference_id="frame_1",
    annotation_id="frame_1_car_box_1",
    metadata={"vehicle_color": "red"},
    track_reference_id="red car",       # <-- This creates a new "red car" track
)

This reference ID is user-defined and does not need to be instantiated before use.

Retrieving Tracks

Tracks are accessed as a list of Track objects through dataset.tracks:

dataset = client.get_dataset(...)
my_tracks = dataset.tracks

Supply and Update Track Metadata

Arbitrary key-value metadata may be stored on the Track level, as well. To update a Track, you can select a Track from the dataset and call the update function.

track_to_update = [track for track in my_tracks if track.reference_id == "red car"][0]

track_to_update.update(new_metadata_1, overwrite_metadata=True)

If overwrite_metadata is set to True, the entire metadata object will be overwritten. This is useful for clearing keys-value pairs. Otherwise, a keywise merge will be performed on the existing metadata.

Retrieve Track Metadata

Metadata can be easily retrieved via the metadata property on the Track object.

track_to_update.metadata

Deleting Tracks

To delete a Track, call delete_tracks from the dataset level, supplying a list of Track reference IDs to delete. Deleting Tracks does not delete the annotations and predictions they point to.

dataset.delete_tracks(["red car", ...])

Integration with Labeling

Scale uses non-unique annotation IDs by default for annotations that point to the same object across multiple frames. The IDs from Labeling, which are in UUIDv4 format, are persisted to Nucleus as Tracks.

Viewing Tracks

Viewing Tracks in Nucleus is available on an on-demand basis. Please contact Scale to enable this feature for your team.