Geometric (Box, Polygon, Line, Keypoints, Cuboid) Predictions
In this guide, we'll use the Python SDK to construct geometric predictions. Each geometric
prediction object in Nucleus has five components:
- Label of the prediction in the class taxonomy, e.g. car or pedestrian
- Geometry to define the position, size, etc. of the box / polygon / line / cuboid
- Reference ID of the item to which to apply the prediction
- Confidence of the model for the prediction
- Metadata, optional key-value pairs pertaining to the prediction, e.g. vehicle_color = blue
- Annotation ID, an optional unique (per dataset item) key to refer to the prediction
- Class PDF, an optional mapping from each class to its corresponding value in the model probability distribution
All geometric prediction types (box, polygon, line, cuboid) share the same structure aside from the geometry. Below, we'll explore how to construct the prediction objects, including geometries, for each type:
BoxPrediction
BoxPrediction
A 2D bounding box based on a given XY position, width, and height.
For more info, see our Python SDK Reference for BoxPredictions
.
from nucleus import BoxPrediction
box_gt = BoxPrediction(
label="car",
x=4, # px between left edge of the box and left border of the image
y=6, # px between top edge of the box and top border of the image
width=12, # width in px
height=18, # height in px
reference_id="image_1",
confidence=0.7 # confidence of the prediction
class_pdf={"car": 0.7, "bus": 0.2, "person": 0.1}, # probability distribution
annotation_id="image_1_car_box_1",
metadata={"vehicle_color": "ruby"}
)
PolygonPrediction
PolygonPrediction
A 2D polygon constructed by connecting 3+ vertices.
For more info, see our Python SDK Reference for PolygonPredictions
.
from nucleus import Point, PolygonPrediction
polygon_gt = PolygonPrediction(
label="bus",
vertices=[ # list of vertices as Point objects
Point(x=100, y=100),
Point(150, 200),
Point(200, 100),
],
reference_id="image_2",
confidence=0.8, # confidence of the prediction
class_pdf={"car": 0.15, "bus": 0.8, "person": 0.05}, # probability distribution
annotation_id="image_2_bus_polygon_1",
metadata={"vehicle_color": "sapphire"}
)
LinePrediction
LinePrediction
A 2D line constructed by connecting 2+ vertices (an unclosed polygon).
For more info, see our Python SDK Reference for LinePredictions
.
from nucleus import Point, LinePrediction
line_gt = LinePrediction(
label="lane",
vertices=[ # list of vertices as Point objects
Point(x=100, y=100),
Point(200, 300),
Point(300, 200),
],
reference_id="image_1",
confidence=0.7, # confidence of the prediction
class_pdf={"lane": 0.7, "median": 0.2, "crosswalk": 0.1}, # probability distribution
annotation_id="image_1_lane_line_1",
metadata={"position": "shoulder"},
)
KeypointsPrediction
KeypointsPrediction
A collection of 2D points that are named and connected via a domain-specific skeleton.
For more info, see our Python SDK Reference for KeypointsPrediction
.
from nucleus import Keypoint, KeypointsPrediction
keypoint_prediction = KeypointsPrediction(
label="face",
keypoints=[
Keypoint(100, 100, visible=True),
Keypoint(200, 300, visible=False),
Keypoint(visible=False),
],
names=["left_eye", "right_eye", "nose"],
skeleton=[[0, 2], [1, 2]],
reference_id="image_1",
confidence=0.8, # confidence of the prediction
annotation_id="image_1_face_1",
metadata={"visibility": "partial"},
)
CuboidPrediction
CuboidPrediction
A 3D cuboid based on a given position, dimensions, and yaw. Cannot be applied to 2D images, only 3D pointcloud DatasetItems
.
For more info, see our Python SDK Reference for CuboidPredictions
.
from nucleus import Point3D, CuboidPrediction
cuboid_gt = CuboidPrediction(
label="car",
position=Point3D(100, 100, 10), # center of the cuboid
dimensions=Point3D(x=5, y=10, z=5), # length (x), width (y), height (z)
yaw=0, # rotation in rad about the Z axis
reference_id="pointcloud_1",
confidence=0.8, # confidence of the prediction
class_pdf={"car": 0.8, "bus": 0.1, "person": 0.1}, # probability distribution
annotation_id="pointcloud_1_car_cuboid_1",
metadata={"vehicle_color": "emerald"}
)
Updated over 2 years ago