Ultralytics Explorer API
As of ultralytics>=8.3.12, Ultralytics Explorer has been removed. To use Explorer, install pip install ultralytics==8.3.11. Similar (and expanded) dataset exploration features are available in Ultralytics Platform.
Introduction#
The Explorer API is a Python API for exploring your datasets. It supports filtering and searching your dataset using SQL queries, vector similarity search, and semantic search.
Watch: Ultralytics Explorer API Overview
Installation#
Explorer depends on external libraries for some of its functionality. These are automatically installed when you use Explorer. To manually install these dependencies, use the following command:
pip install "ultralytics[explorer]==8.3.11"Usage#
from ultralytics import Explorer
# Create an Explorer object
explorer = Explorer(data="coco128.yaml", model="yolo11n.pt")
# Create embeddings for your dataset
explorer.create_embeddings_table()
# Search for similar images to a given image/images
df = explorer.get_similar(img="path/to/image.jpg")
# Or search for similar images to a given index/indices
df = explorer.get_similar(idx=0)Embeddings table for a given dataset and model pair is only created once and reused. These use LanceDB under the hood, which scales on-disk, so you can create and reuse embeddings for large datasets like COCO without running out of memory.
To force an update of the embeddings table, pass force=True to the create_embeddings_table method.
You can directly access the LanceDB table object to perform advanced analysis. Learn more about it in the Working with Embeddings Table section.
1. Similarity Search#
Similarity search, also known as vector search, is a technique for finding images similar to a given image. It is based on the idea that similar images will have similar embeddings. Once the embeddings table is built, you can run similarity search in any of the following ways:
- On a given index or list of indices in the dataset:
exp.get_similar(idx=[1,10], limit=10) - On any image or list of images not in the dataset:
exp.get_similar(img=["path/to/img1", "path/to/img2"], limit=10)
In case of multiple inputs, the aggregate of their embeddings is used.
You get a pandas DataFrame with the limit number of most similar data points to the input, along with their distance in the embedding space. You can use this dataset to perform further filtering.
from ultralytics import Explorer
# create an Explorer object
exp = Explorer(data="coco128.yaml", model="yolo11n.pt")
exp.create_embeddings_table()
similar = exp.get_similar(img="https://ultralytics.com/images/bus.jpg", limit=10)
print(similar.head())
# Search using multiple images
similar = exp.get_similar(
img=["https://ultralytics.com/images/bus.jpg", "https://ultralytics.com/images/bus.jpg"],
limit=10,
)
print(similar.head())Plotting Similar Images#
You can also plot the similar images using the plot_similar method. This method takes the same arguments as get_similar and plots the similar images in a grid.
from ultralytics import Explorer
# create an Explorer object
exp = Explorer(data="coco128.yaml", model="yolo11n.pt")
exp.create_embeddings_table()
plt = exp.plot_similar(img="https://ultralytics.com/images/bus.jpg", limit=10)
plt.show()2. Ask AI (Natural Language Querying)#
This feature lets you filter your dataset using natural language, without writing SQL. The AI-powered query generator converts your prompt into a query and returns matching results. For example, you can ask: "show me 100 images with exactly one person and 2 dogs. There can be other objects too" and it will generate the query and show you those results. Note: This feature uses LLMs, so results are probabilistic and may be inaccurate.
from ultralytics.data.explorer import plot_query_result
from ultralytics import Explorer
# create an Explorer object
exp = Explorer(data="coco128.yaml", model="yolo11n.pt")
exp.create_embeddings_table()
df = exp.ask_ai("show me 100 images with exactly one person and 2 dogs. There can be other objects too")
print(df.head())
# plot the results
plt = plot_query_result(df)
plt.show()3. SQL Querying#
You can run SQL queries on your dataset using the sql_query method. This method takes a SQL query as input and returns a pandas DataFrame with the results.
from ultralytics import Explorer
# create an Explorer object
exp = Explorer(data="coco128.yaml", model="yolo11n.pt")
exp.create_embeddings_table()
df = exp.sql_query("WHERE labels LIKE '%person%' AND labels LIKE '%dog%'")
print(df.head())Plotting SQL Query Results#
You can also plot the results of a SQL query using the plot_sql_query method. This method takes the same arguments as sql_query and plots the results in a grid.
from ultralytics import Explorer
# create an Explorer object
exp = Explorer(data="coco128.yaml", model="yolo11n.pt")
exp.create_embeddings_table()
# plot the SQL Query
exp.plot_sql_query("WHERE labels LIKE '%person%' AND labels LIKE '%dog%' LIMIT 10")4. Working with Embeddings Table#
You can also work with the embeddings table directly. Once the embeddings table is created, you can access it using Explorer.table.
Explorer works on LanceDB tables internally. You can access this table directly, using Explorer.table object and run raw queries, push down pre- and post-filters, etc.
from ultralytics import Explorer
exp = Explorer()
exp.create_embeddings_table()
table = exp.tableHere are some examples of what you can do with the table:
Get raw Embeddings#
from ultralytics import Explorer
exp = Explorer()
exp.create_embeddings_table()
table = exp.table
embeddings = table.to_pandas()["vector"]
print(embeddings)Advanced Querying with pre- and post-filters#
from ultralytics import Explorer
exp = Explorer(model="yolo11n.pt")
exp.create_embeddings_table()
table = exp.table
# Dummy embedding
embedding = [i for i in range(256)]
rs = table.search(embedding).metric("cosine").where("").limit(10)Create Vector Index#
When using large datasets, you can also create a dedicated vector index for faster querying. This is done using the create_index method on LanceDB table.
table.create_index(num_partitions=..., num_sub_vectors=...)5. Embeddings Applications#
You can use the embeddings table to perform a variety of exploratory analysis. Here are some examples:
Similarity Index#
Explorer comes with a similarity_index operation:
- It tries to estimate how similar each data point is with the rest of the dataset.
- It does that by counting how many image embeddings lie closer than
max_distto the current image in the generated embedding space, consideringtop_ksimilar images at a time.
It returns a pandas DataFrame with the following columns:
idx: Index of the image in the datasetim_file: Path to the image filecount: Number of images in the dataset that are closer thanmax_distto the current imagesim_im_files: List of paths to thecountsimilar images
For a given dataset, model, max_dist & top_k the similarity index once generated will be reused. If your dataset has changed, or you need to regenerate the similarity index, pass force=True.
from ultralytics import Explorer
exp = Explorer()
exp.create_embeddings_table()
sim_idx = exp.similarity_index()You can use the similarity index to build custom conditions for filtering the dataset. For example, the following code lists images that have more than 30 similar images in the dataset, which are good candidates for near-duplicate removal:
import numpy as np
sim_count = np.array(sim_idx["count"])
sim_idx["im_file"][sim_count > 30]Visualize Embedding Space#
You can also visualize the embedding space using the plotting tool of your choice. Here is a simple example that reduces the embeddings with PCA and plots them with Matplotlib:
import matplotlib.pyplot as plt
from sklearn.decomposition import PCA
# Reduce dimensions using PCA to 3 components for visualization in 3D
pca = PCA(n_components=3)
reduced_data = pca.fit_transform(embeddings)
# Create a 3D scatter plot using Matplotlib Axes3D
fig = plt.figure(figsize=(8, 6))
ax = fig.add_subplot(111, projection="3d")
# Scatter plot
ax.scatter(reduced_data[:, 0], reduced_data[:, 1], reduced_data[:, 2], alpha=0.5)
ax.set_title("3D Scatter Plot of Reduced 256-Dimensional Data (PCA)")
ax.set_xlabel("Component 1")
ax.set_ylabel("Component 2")
ax.set_zlabel("Component 3")
plt.show()Start creating your own CV dataset exploration reports using the Explorer API. For inspiration, check out the VOC Exploration Example.
Apps Built Using Ultralytics Explorer#
The Explorer GUI is built on the Explorer API.
FAQ#
The Ultralytics Explorer API is designed for dataset exploration. It lets you filter and search datasets using SQL queries, vector similarity search, and semantic search, and it scales to large datasets for computer vision tasks with Ultralytics models.
To install the Ultralytics Explorer API along with its dependencies, use the following command:
pip install "ultralytics[explorer]==8.3.11"This will automatically install all necessary external libraries for the Explorer API functionality. For additional setup details, refer to the installation section of our documentation.
You can use the Ultralytics Explorer API to perform similarity searches by creating an embeddings table and querying it for similar images. Here's a basic example:
from ultralytics import Explorer # Create an Explorer object explorer = Explorer(data="coco128.yaml", model="yolo11n.pt") explorer.create_embeddings_table() # Search for similar images to a given image similar_images_df = explorer.get_similar(img="path/to/image.jpg") print(similar_images_df.head())For more details, please visit the Similarity Search section.
LanceDB, used under the hood by Ultralytics Explorer, provides scalable, on-disk embeddings tables. This ensures that you can create and reuse embeddings for large datasets like COCO without running out of memory. These tables are only created once and can be reused, enhancing efficiency in data handling.
The Ask AI feature allows users to filter datasets using natural language queries. This feature leverages LLMs to convert these queries into SQL queries behind the scenes. Here's an example:
from ultralytics import Explorer # Create an Explorer object explorer = Explorer(data="coco128.yaml", model="yolo11n.pt") explorer.create_embeddings_table() # Query with natural language query_result = explorer.ask_ai("show me 100 images with exactly one person and 2 dogs. There can be other objects too") print(query_result.head())For more examples, check out the Ask AI section.
Comments2
Ask a question, share your results, or help someone else.
Jie LiuI have downloaded
ultralytics-8.3.118 github zip packageand install it withpip install -e .. Then I usepip install ultralytics[explorer]to install explorer, but I got a error as followedRequirement already satisfied: ultralytics[explorer] in /data/miniconda3/envs/app/lib/python3.10/site-packages (8.3.118) WARNING: ultralytics 8.3.118 does not provide the extra 'explorer' Requirement already satisfied: numpy<=2.1.1,>=1.23.0 in /data/miniconda3/envs/app/lib/python3.10/site-packages (from ultralytics[explorer]) (1.26.4) Requirement already satisfied: matplotlib>=3.3.0 in /data/miniconda3/envs/app/lib/python3.10/site-packages (from ultralytics[explorer]) (3.10.1) Requirement already satisfied: opencv-python>=4.6.0 in /data/miniconda3/envs/app/lib/python3.10/site-packages (from ultralytics[explorer]) (4.11.0.86) Requirement already satisfied: pillow>=7.1.2 in /data/miniconda3/envs/app/lib/python3.10/site-packages (from ultralytics[explorer]) (11.2.1) Requirement already satisfied: pyyaml>=5.3.1 in /data/miniconda3/envs/app/lib/python3.10/site-packages (from ultralytics[explorer]) (6.0.2) Requirement already satisfied: requests>=2.23.0 in /data/miniconda3/envs/app/lib/python3.10/site-packages (from ultralytics[explorer]) (2.32.3) Requirement already satisfied: scipy>=1.4.1 in /data/miniconda3/envs/app/lib/python3.10/site-packages (from ultralytics[explorer]) (1.15.2) Requirement already satisfied: torch>=1.8.0 in /data/miniconda3/envs/app/lib/python3.10/site-packages (from ultralytics[explorer]) (2.3.1) Requirement already satisfied: torchvision>=0.9.0 in /data/miniconda3/envs/app/lib/python3.10/site-packages (from ultralytics[explorer]) (0.18.1) Requirement already satisfied: tqdm>=4.64.0 in /data/miniconda3/envs/app/lib/python3.10/site-packages (from ultralytics[explorer]) (4.67.1) Requirement already satisfied: psutil in /data/miniconda3/envs/app/lib/python3.10/site-packages (from ultralytics[explorer]) (7.0.0) Requirement already satisfied: py-cpuinfo in /data/miniconda3/envs/app/lib/python3.10/site-packages (from ultralytics[explorer]) (9.0.0) Requirement already satisfied: pandas>=1.1.4 in /data/miniconda3/envs/app/lib/python3.10/site-packages (from ultralytics[explorer]) (2.2.3) Requirement already satisfied: seaborn>=0.11.0 in /data/miniconda3/envs/app/lib/python3.10/site-packages (from ultralytics[explorer]) (0.13.2) Requirement already satisfied: ultralytics-thop>=2.0.0 in /data/miniconda3/envs/app/lib/python3.10/site-packages (from ultralytics[explorer]) (2.0.14) Requirement already satisfied: contourpy>=1.0.1 in /data/miniconda3/envs/app/lib/python3.10/site-packages (from matplotlib>=3.3.0->ultralytics[explorer]) (1.3.2) Requirement already satisfied: cycler>=0.10 in /data/miniconda3/envs/app/lib/python3.10/site-packages (from matplotlib>=3.3.0->ultralytics[explorer]) (0.12.1) Requirement already satisfied: fonttools>=4.22.0 in /data/miniconda3/envs/app/lib/python3.10/site-packages (from matplotlib>=3.3.0->ultralytics[explorer]) (4.57.0) Requirement already satisfied: kiwisolver>=1.3.1 in /data/miniconda3/envs/app/lib/python3.10/site-packages (from matplotlib>=3.3.0->ultralytics[explorer]) (1.4.8) Requirement already satisfied: packaging>=20.0 in /data/miniconda3/envs/app/lib/python3.10/site-packages (from matplotlib>=3.3.0->ultralytics[explorer]) (24.2) Requirement already satisfied: pyparsing>=2.3.1 in /data/miniconda3/envs/app/lib/python3.10/site-packages (from matplotlib>=3.3.0->ultralytics[explorer]) (3.2.3) Requirement already satisfied: python-dateutil>=2.7 in /data/miniconda3/envs/app/lib/python3.10/site-packages (from matplotlib>=3.3.0->ultralytics[explorer]) (2.9.0.post0) Requirement already satisfied: pytz>=2020.1 in /data/miniconda3/envs/app/lib/python3.10/site-packages (from pandas>=1.1.4->ultralytics[explorer]) (2022.7.1) Requirement already satisfied: tzdata>=2022.7 in /data/miniconda3/envs/app/lib/python3.10/site-packages (from pandas>=1.1.4->ultralytics[explorer]) (2025.2) Requirement already satisfied: charset-normalizer<4,>=2 in /data/miniconda3/envs/app/lib/python3.10/site-packages (from requests>=2.23.0->ultralytics[explorer]) (3.4.1) Requirement already satisfied: idna<4,>=2.5 in /data/miniconda3/envs/app/lib/python3.10/site-packages (from requests>=2.23.0->ultralytics[explorer]) (3.10) Requirement already satisfied: urllib3<3,>=1.21.1 in /data/miniconda3/envs/app/lib/python3.10/site-packages (from requests>=2.23.0->ultralytics[explorer]) (1.26.20) Requirement already satisfied: certifi>=2017.4.17 in /data/miniconda3/envs/app/lib/python3.10/site-packages (from requests>=2.23.0->ultralytics[explorer]) (2025.4.26) Requirement already satisfied: filelock in /data/miniconda3/envs/app/lib/python3.10/site-packages (from torch>=1.8.0->ultralytics[explorer]) (3.18.0) Requirement already satisfied: typing-extensions>=4.8.0 in /data/miniconda3/envs/app/lib/python3.10/site-packages (from torch>=1.8.0->ultralytics[explorer]) (4.13.2) Requirement already satisfied: sympy in /data/miniconda3/envs/app/lib/python3.10/site-packages (from torch>=1.8.0->ultralytics[explorer]) (1.13.3) Requirement already satisfied: networkx in /data/miniconda3/envs/app/lib/python3.10/site-packages (from torch>=1.8.0->ultralytics[explorer]) (3.4.2) Requirement already satisfied: jinja2 in /data/miniconda3/envs/app/lib/python3.10/site-packages (from torch>=1.8.0->ultralytics[explorer]) (3.1.6) Requirement already satisfied: fsspec in /data/miniconda3/envs/app/lib/python3.10/site-packages (from torch>=1.8.0->ultralytics[explorer]) (2025.3.2) Requirement already satisfied: nvidia-cuda-nvrtc-cu12==12.1.105 in /data/miniconda3/envs/app/lib/python3.10/site-packages (from torch>=1.8.0->ultralytics[explorer]) (12.1.105) Requirement already satisfied: nvidia-cuda-runtime-cu12==12.1.105 in /data/miniconda3/envs/app/lib/python3.10/site-packages (from torch>=1.8.0->ultralytics[explorer]) (12.1.105) Requirement already satisfied: nvidia-cuda-cupti-cu12==12.1.105 in /data/miniconda3/envs/app/lib/python3.10/site-packages (from torch>=1.8.0->ultralytics[explorer]) (12.1.105) Requirement already satisfied: nvidia-cudnn-cu12==8.9.2.26 in /data/miniconda3/envs/app/lib/python3.10/site-packages (from torch>=1.8.0->ultralytics[explorer]) (8.9.2.26) Requirement already satisfied: nvidia-cublas-cu12==12.1.3.1 in /data/miniconda3/envs/app/lib/python3.10/site-packages (from torch>=1.8.0->ultralytics[explorer]) (12.1.3.1) Requirement already satisfied: nvidia-cufft-cu12==11.0.2.54 in /data/miniconda3/envs/app/lib/python3.10/site-packages (from torch>=1.8.0->ultralytics[explorer]) (11.0.2.54) Requirement already satisfied: nvidia-curand-cu12==10.3.2.106 in /data/miniconda3/envs/app/lib/python3.10/site-packages (from torch>=1.8.0->ultralytics[explorer]) (10.3.2.106) Requirement already satisfied: nvidia-cusolver-cu12==11.4.5.107 in /data/miniconda3/envs/app/lib/python3.10/site-packages (from torch>=1.8.0->ultralytics[explorer]) (11.4.5.107) Requirement already satisfied: nvidia-cusparse-cu12==12.1.0.106 in /data/miniconda3/envs/app/lib/python3.10/site-packages (from torch>=1.8.0->ultralytics[explorer]) (12.1.0.106) Requirement already satisfied: nvidia-nccl-cu12==2.20.5 in /data/miniconda3/envs/app/lib/python3.10/site-packages (from torch>=1.8.0->ultralytics[explorer]) (2.20.5) Requirement already satisfied: nvidia-nvtx-cu12==12.1.105 in /data/miniconda3/envs/app/lib/python3.10/site-packages (from torch>=1.8.0->ultralytics[explorer]) (12.1.105) Requirement already satisfied: triton==2.3.1 in /data/miniconda3/envs/app/lib/python3.10/site-packages (from torch>=1.8.0->ultralytics[explorer]) (2.3.1) Requirement already satisfied: nvidia-nvjitlink-cu12 in /data/miniconda3/envs/app/lib/python3.10/site-packages (from nvidia-cusolver-cu12==11.4.5.107->torch>=1.8.0->ultralytics[explorer]) (12.8.93) Requirement already satisfied: six>=1.5 in /data/miniconda3/envs/app/lib/python3.10/site-packages (from python-dateutil>=2.7->matplotlib>=3.3.0->ultralytics[explorer]) (1.17.0) Requirement already satisfied: MarkupSafe>=2.0 in /data/miniconda3/envs/app/lib/python3.10/site-packages (from jinja2->torch>=1.8.0->ultralytics[explorer]) (3.0.2) Requirement already satisfied: mpmath<1.4,>=1.1.0 in /data/miniconda3/envs/app/lib/python3.10/site-packages (from sympy->torch>=1.8.0->ultralytics[explorer]) (1.3.0)when run code
from ultralytics import Explorer # Create an Explorer object explorer = Explorer(data="coco128.yaml", model="yolo11n.pt") # Create embeddings for your dataset explorer.create_embeddings_table() # Search for similar images to a given image/images dataframe = explorer.get_similar(img="path/to/image.jpg") # Or search for similar images to a given index/indices dataframe = explorer.get_similar(idx=0)The error has arisen
Traceback (most recent call last): File "/data/liujie/data/ultralytics-8.3.13/tutorial/debug/explorer.py", line 1, in <module> from ultralytics import Explorer ImportError: cannot import name 'Explorer' from 'ultralytics' (/data/liujie/data/ultralytics-8.3.118/ultralytics/__init__.py)please help me to install and use explorer
Glenn JocherUltralytics(edited)Hi @keeper-jie,
The issue you're experiencing is because Ultralytics Explorer has been removed since version 8.3.12, and you're using version 8.3.118. That's why you're seeing the warning that the 'explorer' extra is not provided and why you can't import the Explorer class.
As noted in our documentation, we've moved this functionality to Ultralytics Platform, our platform that offers similar and enhanced capabilities for exploring, visualizing, and managing your datasets without writing code.
If you specifically need the Explorer API functionality in your code, you would need to downgrade to a version prior to 8.3.12. However, we recommend trying Ultralytics Platform instead, as it provides an improved experience with all the dataset exploration features you're looking for.
Let me know if you need any further assistance with Ultralytics Platform or if you have any other questions.