Overview

Autotag, Similarity Search, and Natural Language Search* in Nucleus are all powered through the use of embeddings. Though Nucleus normally generates these embeddings internally as you upload items and objects, it is also possible to supply your own custom embeddings to Nucleus. This allows the features mentioned above to be more fine tuned for your dataset's particular use case.

Indexing Status

A set of embeddings on a dataset is abstracted as an index. Each index will only have at most one embedding per item. All datasets automatically have an internal index with Nucleus generated embeddings for images.

Switching Indexes

By default, the features listed above are powered by internal, Nucleus generated embeddings. You can use this endpoint to switch between using internal or custom embeddings.

Custom Image Embeddings

To upload your own custom image embeddings, use the custom image embedding API endpoint. You can call this endpoint multiple times to write embeddings for new images that have been added to your dataset.

However, modifying existing embeddings is currently not supported. If you wish to replace existing embeddings, you must first delete all your custom image embeddings via this endpoint.

Custom Object Embeddings

Custom object embeddings may be uploaded for existing or new ground truth and model prediction annotations. To do so, simply set the embedding_vector parameter when constructing your ground truth and model prediction classes in the Python client. You can then upload your annotations normally. If the annotations already exist, make sure to set update = True when uploading through the client.

However, modifying existing embeddings is currently not supported. If you wish to replace existing embeddings, you must first delete all your custom object embeddings via this endpoint.

^{*Natural Language Search does not support the use of custom embeddings.}