Description

Nucleus goes beyond single model evaluation to allow you to do rich comparison between multiple models. You can compare model performance on various evaluation metrics and further break it down on arbitrary metadata dimensions (annotations, predictions, item metadata etc.). Through this workflow you can establish if and where a new model is better/worse than a baseline and thus do targeted improvement.

Pre-reqs: Private Dataset, Annotations, Predictions (at least two models)

Steps

1. Navigate to the “Model Comparison” page using the sidebar
2. Select a dataset using the dropdown on the top e.g. “BDD”
3. Select Model 1 & Model 2 which you want to compare
4. Type a query to limit the results to a specific subset of interest e.g. “slice = X”
5. Use the charts to view performance comparison e.g. “Class Distribution”
6. Click on any chart e.g. bar chart and view the data in the Nucleus grid view