Description

With this workflow you can search for arbitrary objects, scenarios or patterns in unlabeled data using natural language queries. For example it enables you to find specific rare edge cases without manual screening.

Pre-reqs: Dataset with embeddings

Steps

1. Open an indexed dataset e.g. BDD
2. Type in a natural language query in the search bar e.g. photo of traffic light
3. Nucleus will show you the closest matching items

Typically natural language queries structured like “a photo of X” perform better. Interesting queries which you can try running include “photo of an intersection”, “photo of bicyclists”, “photo of police cars” etc. Once you find a sample image of interest you can easily gather more such samples using autotag.