Get Started

Ultralytics Explorer API

Community Note ⚠️

As of ultralytics>=8.3.12, Ultralytics Explorer has been removed. To use Explorer, install pip install ultralytics==8.3.11. Similar (and expanded) dataset exploration features are available in Ultralytics Platform.

Introduction#

The Explorer API is a Python API for exploring your datasets. It supports filtering and searching your dataset using SQL queries, vector similarity search, and semantic search.



Watch: Ultralytics Explorer API Overview

Installation#

Explorer depends on external libraries for some of its functionality. These are automatically installed when you use Explorer. To manually install these dependencies, use the following command:

pip install "ultralytics[explorer]==8.3.11"

Usage#

from ultralytics import Explorer

# Create an Explorer object
explorer = Explorer(data="coco128.yaml", model="yolo11n.pt")

# Create embeddings for your dataset
explorer.create_embeddings_table()

# Search for similar images to a given image/images
df = explorer.get_similar(img="path/to/image.jpg")

# Or search for similar images to a given index/indices
df = explorer.get_similar(idx=0)
Note

Embeddings table for a given dataset and model pair is only created once and reused. These use LanceDB under the hood, which scales on-disk, so you can create and reuse embeddings for large datasets like COCO without running out of memory.

To force an update of the embeddings table, pass force=True to the create_embeddings_table method.

You can directly access the LanceDB table object to perform advanced analysis. Learn more about it in the Working with Embeddings Table section.

Similarity search, also known as vector search, is a technique for finding images similar to a given image. It is based on the idea that similar images will have similar embeddings. Once the embeddings table is built, you can run similarity search in any of the following ways:

  • On a given index or list of indices in the dataset: exp.get_similar(idx=[1,10], limit=10)
  • On any image or list of images not in the dataset: exp.get_similar(img=["path/to/img1", "path/to/img2"], limit=10)

In case of multiple inputs, the aggregate of their embeddings is used.

You get a pandas DataFrame with the limit number of most similar data points to the input, along with their distance in the embedding space. You can use this dataset to perform further filtering.

Semantic Search
from ultralytics import Explorer

# create an Explorer object
exp = Explorer(data="coco128.yaml", model="yolo11n.pt")
exp.create_embeddings_table()

similar = exp.get_similar(img="https://ultralytics.com/images/bus.jpg", limit=10)
print(similar.head())

# Search using multiple images
similar = exp.get_similar(
    img=["https://ultralytics.com/images/bus.jpg", "https://ultralytics.com/images/bus.jpg"],
    limit=10,
)
print(similar.head())

Plotting Similar Images#

You can also plot the similar images using the plot_similar method. This method takes the same arguments as get_similar and plots the similar images in a grid.

Plotting Similar Images
from ultralytics import Explorer

# create an Explorer object
exp = Explorer(data="coco128.yaml", model="yolo11n.pt")
exp.create_embeddings_table()

plt = exp.plot_similar(img="https://ultralytics.com/images/bus.jpg", limit=10)
plt.show()

2. Ask AI (Natural Language Querying)#

This feature lets you filter your dataset using natural language, without writing SQL. The AI-powered query generator converts your prompt into a query and returns matching results. For example, you can ask: "show me 100 images with exactly one person and 2 dogs. There can be other objects too" and it will generate the query and show you those results. Note: This feature uses LLMs, so results are probabilistic and may be inaccurate.

Ask AI
from ultralytics.data.explorer import plot_query_result

from ultralytics import Explorer

# create an Explorer object
exp = Explorer(data="coco128.yaml", model="yolo11n.pt")
exp.create_embeddings_table()

df = exp.ask_ai("show me 100 images with exactly one person and 2 dogs. There can be other objects too")
print(df.head())

# plot the results
plt = plot_query_result(df)
plt.show()

3. SQL Querying#

You can run SQL queries on your dataset using the sql_query method. This method takes a SQL query as input and returns a pandas DataFrame with the results.

SQL Query
from ultralytics import Explorer

# create an Explorer object
exp = Explorer(data="coco128.yaml", model="yolo11n.pt")
exp.create_embeddings_table()

df = exp.sql_query("WHERE labels LIKE '%person%' AND labels LIKE '%dog%'")
print(df.head())

Plotting SQL Query Results#

You can also plot the results of a SQL query using the plot_sql_query method. This method takes the same arguments as sql_query and plots the results in a grid.

Plotting SQL Query Results
from ultralytics import Explorer

# create an Explorer object
exp = Explorer(data="coco128.yaml", model="yolo11n.pt")
exp.create_embeddings_table()

# plot the SQL Query
exp.plot_sql_query("WHERE labels LIKE '%person%' AND labels LIKE '%dog%' LIMIT 10")

4. Working with Embeddings Table#

You can also work with the embeddings table directly. Once the embeddings table is created, you can access it using Explorer.table.

Tip

Explorer works on LanceDB tables internally. You can access this table directly, using Explorer.table object and run raw queries, push down pre- and post-filters, etc.

from ultralytics import Explorer

exp = Explorer()
exp.create_embeddings_table()
table = exp.table

Here are some examples of what you can do with the table:

Get raw Embeddings#

Example
from ultralytics import Explorer

exp = Explorer()
exp.create_embeddings_table()
table = exp.table

embeddings = table.to_pandas()["vector"]
print(embeddings)

Advanced Querying with pre- and post-filters#

Example
from ultralytics import Explorer

exp = Explorer(model="yolo11n.pt")
exp.create_embeddings_table()
table = exp.table

# Dummy embedding
embedding = [i for i in range(256)]
rs = table.search(embedding).metric("cosine").where("").limit(10)

Create Vector Index#

When using large datasets, you can also create a dedicated vector index for faster querying. This is done using the create_index method on LanceDB table.

table.create_index(num_partitions=..., num_sub_vectors=...)

5. Embeddings Applications#

You can use the embeddings table to perform a variety of exploratory analysis. Here are some examples:

Similarity Index#

Explorer comes with a similarity_index operation:

  • It tries to estimate how similar each data point is with the rest of the dataset.
  • It does that by counting how many image embeddings lie closer than max_dist to the current image in the generated embedding space, considering top_k similar images at a time.

It returns a pandas DataFrame with the following columns:

  • idx: Index of the image in the dataset
  • im_file: Path to the image file
  • count: Number of images in the dataset that are closer than max_dist to the current image
  • sim_im_files: List of paths to the count similar images
Tip

For a given dataset, model, max_dist & top_k the similarity index once generated will be reused. If your dataset has changed, or you need to regenerate the similarity index, pass force=True.

Similarity Index
from ultralytics import Explorer

exp = Explorer()
exp.create_embeddings_table()

sim_idx = exp.similarity_index()

You can use the similarity index to build custom conditions for filtering the dataset. For example, the following code lists images that have more than 30 similar images in the dataset, which are good candidates for near-duplicate removal:

import numpy as np

sim_count = np.array(sim_idx["count"])
sim_idx["im_file"][sim_count > 30]

Visualize Embedding Space#

You can also visualize the embedding space using the plotting tool of your choice. Here is a simple example that reduces the embeddings with PCA and plots them with Matplotlib:

import matplotlib.pyplot as plt
from sklearn.decomposition import PCA

# Reduce dimensions using PCA to 3 components for visualization in 3D
pca = PCA(n_components=3)
reduced_data = pca.fit_transform(embeddings)

# Create a 3D scatter plot using Matplotlib Axes3D
fig = plt.figure(figsize=(8, 6))
ax = fig.add_subplot(111, projection="3d")

# Scatter plot
ax.scatter(reduced_data[:, 0], reduced_data[:, 1], reduced_data[:, 2], alpha=0.5)
ax.set_title("3D Scatter Plot of Reduced 256-Dimensional Data (PCA)")
ax.set_xlabel("Component 1")
ax.set_ylabel("Component 2")
ax.set_zlabel("Component 3")

plt.show()

Start creating your own CV dataset exploration reports using the Explorer API. For inspiration, check out the VOC Exploration Example.

Apps Built Using Ultralytics Explorer#

The Explorer GUI is built on the Explorer API.

FAQ#

  • The Ultralytics Explorer API is designed for dataset exploration. It lets you filter and search datasets using SQL queries, vector similarity search, and semantic search, and it scales to large datasets for computer vision tasks with Ultralytics models.

  • To install the Ultralytics Explorer API along with its dependencies, use the following command:

    pip install "ultralytics[explorer]==8.3.11"

    This will automatically install all necessary external libraries for the Explorer API functionality. For additional setup details, refer to the installation section of our documentation.

  • You can use the Ultralytics Explorer API to perform similarity searches by creating an embeddings table and querying it for similar images. Here's a basic example:

    from ultralytics import Explorer
    
    # Create an Explorer object
    explorer = Explorer(data="coco128.yaml", model="yolo11n.pt")
    explorer.create_embeddings_table()
    
    # Search for similar images to a given image
    similar_images_df = explorer.get_similar(img="path/to/image.jpg")
    print(similar_images_df.head())

    For more details, please visit the Similarity Search section.

  • LanceDB, used under the hood by Ultralytics Explorer, provides scalable, on-disk embeddings tables. This ensures that you can create and reuse embeddings for large datasets like COCO without running out of memory. These tables are only created once and can be reused, enhancing efficiency in data handling.

  • The Ask AI feature allows users to filter datasets using natural language queries. This feature leverages LLMs to convert these queries into SQL queries behind the scenes. Here's an example:

    from ultralytics import Explorer
    
    # Create an Explorer object
    explorer = Explorer(data="coco128.yaml", model="yolo11n.pt")
    explorer.create_embeddings_table()
    
    # Query with natural language
    query_result = explorer.ask_ai("show me 100 images with exactly one person and 2 dogs. There can be other objects too")
    print(query_result.head())

    For more examples, check out the Ask AI section.

Comments2

Ask a question, share your results, or help someone else.

  1. Jie Liu

    I have downloaded ultralytics-8.3.118 github zip package and install it with pip install -e .. Then I use pip install ultralytics[explorer] to install explorer, but I got a error as followed

    Requirement already satisfied: ultralytics[explorer] in /data/miniconda3/envs/app/lib/python3.10/site-packages (8.3.118)
    WARNING: ultralytics 8.3.118 does not provide the extra 'explorer'
    Requirement already satisfied: numpy<=2.1.1,>=1.23.0 in /data/miniconda3/envs/app/lib/python3.10/site-packages (from ultralytics[explorer]) (1.26.4)
    Requirement already satisfied: matplotlib>=3.3.0 in /data/miniconda3/envs/app/lib/python3.10/site-packages (from ultralytics[explorer]) (3.10.1)
    Requirement already satisfied: opencv-python>=4.6.0 in /data/miniconda3/envs/app/lib/python3.10/site-packages (from ultralytics[explorer]) (4.11.0.86)
    Requirement already satisfied: pillow>=7.1.2 in /data/miniconda3/envs/app/lib/python3.10/site-packages (from ultralytics[explorer]) (11.2.1)
    Requirement already satisfied: pyyaml>=5.3.1 in /data/miniconda3/envs/app/lib/python3.10/site-packages (from ultralytics[explorer]) (6.0.2)
    Requirement already satisfied: requests>=2.23.0 in /data/miniconda3/envs/app/lib/python3.10/site-packages (from ultralytics[explorer]) (2.32.3)
    Requirement already satisfied: scipy>=1.4.1 in /data/miniconda3/envs/app/lib/python3.10/site-packages (from ultralytics[explorer]) (1.15.2)
    Requirement already satisfied: torch>=1.8.0 in /data/miniconda3/envs/app/lib/python3.10/site-packages (from ultralytics[explorer]) (2.3.1)
    Requirement already satisfied: torchvision>=0.9.0 in /data/miniconda3/envs/app/lib/python3.10/site-packages (from ultralytics[explorer]) (0.18.1)
    Requirement already satisfied: tqdm>=4.64.0 in /data/miniconda3/envs/app/lib/python3.10/site-packages (from ultralytics[explorer]) (4.67.1)
    Requirement already satisfied: psutil in /data/miniconda3/envs/app/lib/python3.10/site-packages (from ultralytics[explorer]) (7.0.0)
    Requirement already satisfied: py-cpuinfo in /data/miniconda3/envs/app/lib/python3.10/site-packages (from ultralytics[explorer]) (9.0.0)
    Requirement already satisfied: pandas>=1.1.4 in /data/miniconda3/envs/app/lib/python3.10/site-packages (from ultralytics[explorer]) (2.2.3)
    Requirement already satisfied: seaborn>=0.11.0 in /data/miniconda3/envs/app/lib/python3.10/site-packages (from ultralytics[explorer]) (0.13.2)
    Requirement already satisfied: ultralytics-thop>=2.0.0 in /data/miniconda3/envs/app/lib/python3.10/site-packages (from ultralytics[explorer]) (2.0.14)
    Requirement already satisfied: contourpy>=1.0.1 in /data/miniconda3/envs/app/lib/python3.10/site-packages (from matplotlib>=3.3.0->ultralytics[explorer]) (1.3.2)
    Requirement already satisfied: cycler>=0.10 in /data/miniconda3/envs/app/lib/python3.10/site-packages (from matplotlib>=3.3.0->ultralytics[explorer]) (0.12.1)
    Requirement already satisfied: fonttools>=4.22.0 in /data/miniconda3/envs/app/lib/python3.10/site-packages (from matplotlib>=3.3.0->ultralytics[explorer]) (4.57.0)
    Requirement already satisfied: kiwisolver>=1.3.1 in /data/miniconda3/envs/app/lib/python3.10/site-packages (from matplotlib>=3.3.0->ultralytics[explorer]) (1.4.8)
    Requirement already satisfied: packaging>=20.0 in /data/miniconda3/envs/app/lib/python3.10/site-packages (from matplotlib>=3.3.0->ultralytics[explorer]) (24.2)
    Requirement already satisfied: pyparsing>=2.3.1 in /data/miniconda3/envs/app/lib/python3.10/site-packages (from matplotlib>=3.3.0->ultralytics[explorer]) (3.2.3)
    Requirement already satisfied: python-dateutil>=2.7 in /data/miniconda3/envs/app/lib/python3.10/site-packages (from matplotlib>=3.3.0->ultralytics[explorer]) (2.9.0.post0)
    Requirement already satisfied: pytz>=2020.1 in /data/miniconda3/envs/app/lib/python3.10/site-packages (from pandas>=1.1.4->ultralytics[explorer]) (2022.7.1)
    Requirement already satisfied: tzdata>=2022.7 in /data/miniconda3/envs/app/lib/python3.10/site-packages (from pandas>=1.1.4->ultralytics[explorer]) (2025.2)
    Requirement already satisfied: charset-normalizer<4,>=2 in /data/miniconda3/envs/app/lib/python3.10/site-packages (from requests>=2.23.0->ultralytics[explorer]) (3.4.1)
    Requirement already satisfied: idna<4,>=2.5 in /data/miniconda3/envs/app/lib/python3.10/site-packages (from requests>=2.23.0->ultralytics[explorer]) (3.10)
    Requirement already satisfied: urllib3<3,>=1.21.1 in /data/miniconda3/envs/app/lib/python3.10/site-packages (from requests>=2.23.0->ultralytics[explorer]) (1.26.20)
    Requirement already satisfied: certifi>=2017.4.17 in /data/miniconda3/envs/app/lib/python3.10/site-packages (from requests>=2.23.0->ultralytics[explorer]) (2025.4.26)
    Requirement already satisfied: filelock in /data/miniconda3/envs/app/lib/python3.10/site-packages (from torch>=1.8.0->ultralytics[explorer]) (3.18.0)
    Requirement already satisfied: typing-extensions>=4.8.0 in /data/miniconda3/envs/app/lib/python3.10/site-packages (from torch>=1.8.0->ultralytics[explorer]) (4.13.2)
    Requirement already satisfied: sympy in /data/miniconda3/envs/app/lib/python3.10/site-packages (from torch>=1.8.0->ultralytics[explorer]) (1.13.3)
    Requirement already satisfied: networkx in /data/miniconda3/envs/app/lib/python3.10/site-packages (from torch>=1.8.0->ultralytics[explorer]) (3.4.2)
    Requirement already satisfied: jinja2 in /data/miniconda3/envs/app/lib/python3.10/site-packages (from torch>=1.8.0->ultralytics[explorer]) (3.1.6)
    Requirement already satisfied: fsspec in /data/miniconda3/envs/app/lib/python3.10/site-packages (from torch>=1.8.0->ultralytics[explorer]) (2025.3.2)
    Requirement already satisfied: nvidia-cuda-nvrtc-cu12==12.1.105 in /data/miniconda3/envs/app/lib/python3.10/site-packages (from torch>=1.8.0->ultralytics[explorer]) (12.1.105)
    Requirement already satisfied: nvidia-cuda-runtime-cu12==12.1.105 in /data/miniconda3/envs/app/lib/python3.10/site-packages (from torch>=1.8.0->ultralytics[explorer]) (12.1.105)
    Requirement already satisfied: nvidia-cuda-cupti-cu12==12.1.105 in /data/miniconda3/envs/app/lib/python3.10/site-packages (from torch>=1.8.0->ultralytics[explorer]) (12.1.105)
    Requirement already satisfied: nvidia-cudnn-cu12==8.9.2.26 in /data/miniconda3/envs/app/lib/python3.10/site-packages (from torch>=1.8.0->ultralytics[explorer]) (8.9.2.26)
    Requirement already satisfied: nvidia-cublas-cu12==12.1.3.1 in /data/miniconda3/envs/app/lib/python3.10/site-packages (from torch>=1.8.0->ultralytics[explorer]) (12.1.3.1)
    Requirement already satisfied: nvidia-cufft-cu12==11.0.2.54 in /data/miniconda3/envs/app/lib/python3.10/site-packages (from torch>=1.8.0->ultralytics[explorer]) (11.0.2.54)
    Requirement already satisfied: nvidia-curand-cu12==10.3.2.106 in /data/miniconda3/envs/app/lib/python3.10/site-packages (from torch>=1.8.0->ultralytics[explorer]) (10.3.2.106)
    Requirement already satisfied: nvidia-cusolver-cu12==11.4.5.107 in /data/miniconda3/envs/app/lib/python3.10/site-packages (from torch>=1.8.0->ultralytics[explorer]) (11.4.5.107)
    Requirement already satisfied: nvidia-cusparse-cu12==12.1.0.106 in /data/miniconda3/envs/app/lib/python3.10/site-packages (from torch>=1.8.0->ultralytics[explorer]) (12.1.0.106)
    Requirement already satisfied: nvidia-nccl-cu12==2.20.5 in /data/miniconda3/envs/app/lib/python3.10/site-packages (from torch>=1.8.0->ultralytics[explorer]) (2.20.5)
    Requirement already satisfied: nvidia-nvtx-cu12==12.1.105 in /data/miniconda3/envs/app/lib/python3.10/site-packages (from torch>=1.8.0->ultralytics[explorer]) (12.1.105)
    Requirement already satisfied: triton==2.3.1 in /data/miniconda3/envs/app/lib/python3.10/site-packages (from torch>=1.8.0->ultralytics[explorer]) (2.3.1)
    Requirement already satisfied: nvidia-nvjitlink-cu12 in /data/miniconda3/envs/app/lib/python3.10/site-packages (from nvidia-cusolver-cu12==11.4.5.107->torch>=1.8.0->ultralytics[explorer]) (12.8.93)
    Requirement already satisfied: six>=1.5 in /data/miniconda3/envs/app/lib/python3.10/site-packages (from python-dateutil>=2.7->matplotlib>=3.3.0->ultralytics[explorer]) (1.17.0)
    Requirement already satisfied: MarkupSafe>=2.0 in /data/miniconda3/envs/app/lib/python3.10/site-packages (from jinja2->torch>=1.8.0->ultralytics[explorer]) (3.0.2)
    Requirement already satisfied: mpmath<1.4,>=1.1.0 in /data/miniconda3/envs/app/lib/python3.10/site-packages (from sympy->torch>=1.8.0->ultralytics[explorer]) (1.3.0)

    when run code

    from ultralytics import Explorer
    
    # Create an Explorer object
    explorer = Explorer(data="coco128.yaml", model="yolo11n.pt")
    
    # Create embeddings for your dataset
    explorer.create_embeddings_table()
    
    # Search for similar images to a given image/images
    dataframe = explorer.get_similar(img="path/to/image.jpg")
    
    # Or search for similar images to a given index/indices
    dataframe = explorer.get_similar(idx=0)

    The error has arisen

    Traceback (most recent call last):
      File "/data/liujie/data/ultralytics-8.3.13/tutorial/debug/explorer.py", line 1, in <module>
        from ultralytics import Explorer
    ImportError: cannot import name 'Explorer' from 'ultralytics' (/data/liujie/data/ultralytics-8.3.118/ultralytics/__init__.py)

    please help me to install and use explorer

    1. Glenn JocherUltralytics(edited)

      Hi @keeper-jie,

      The issue you're experiencing is because Ultralytics Explorer has been removed since version 8.3.12, and you're using version 8.3.118. That's why you're seeing the warning that the 'explorer' extra is not provided and why you can't import the Explorer class.

      As noted in our documentation, we've moved this functionality to Ultralytics Platform, our platform that offers similar and enhanced capabilities for exploring, visualizing, and managing your datasets without writing code.

      If you specifically need the Explorer API functionality in your code, you would need to downgrade to a version prior to 8.3.12. However, we recommend trying Ultralytics Platform instead, as it provides an improved experience with all the dataset exploration features you're looking for.

      Let me know if you need any further assistance with Ultralytics Platform or if you have any other questions.