Ultralytics Explorer API
自 ultralytics>=8.3.12 起,Ultralytics Explorer 已被移除。要使用 Explorer,请安装 pip install ultralytics==8.3.11。你可以在 Ultralytics Platform 使用类似(且功能更丰富)的数据集探索功能。
简介#
Explorer API 是一个用于探索数据集的 Python API。它支持使用 SQL 查询、向量相似性搜索和语义搜索来筛选和搜索数据集。
观看: Ultralytics Explorer API 概览
安装#
Explorer 的部分功能依赖外部库。使用 Explorer 时,系统会自动安装这些库。你也可以使用以下命令手动安装这些依赖项:
pip install "ultralytics[explorer]==8.3.11"用法#
from ultralytics import Explorer
# 创建一个 Explorer 对象
explorer = Explorer(data="coco128.yaml", model="yolo11n.pt")
# 为数据集创建嵌入向量
explorer.create_embeddings_table()
# 搜索与给定图像相似的图像
df = explorer.get_similar(img="path/to/image.jpg")
# 或者,搜索与给定索引相似的图像
df = explorer.get_similar(idx=0)要强制更新嵌入表,请将 force=True 传递给 create_embeddings_table 方法。
你可以直接访问 LanceDB 表对象以执行高级分析。请参阅使用嵌入向量表部分了解详情。
1. 相似性搜索#
相似性搜索也称为向量搜索,是一种查找与给定图像相似图像的技术。其原理是相似图像具有相似的嵌入。构建嵌入表后,你可以通过以下任一方式运行相似性搜索:
- 针对数据集中的给定索引或索引列表:
exp.get_similar(idx=[1,10], limit=10) - 针对数据集之外的任意图像或图像列表:
exp.get_similar(img=["path/to/img1", "path/to/img2"], limit=10)
如果有多个输入,则使用它们嵌入向量的聚合结果。
你会获得一个 pandas DataFrame,其中包含与输入最相似的 limit 个数据点及其在嵌入空间中的距离。你可以使用此数据集进行进一步筛选。
from ultralytics import Explorer
# 创建一个 Explorer 对象
exp = Explorer(data="coco128.yaml", model="yolo11n.pt")
exp.create_embeddings_table()
similar = exp.get_similar(img="https://ultralytics.com/images/bus.jpg", limit=10)
print(similar.head())
# 使用多张图像进行搜索
similar = exp.get_similar(
img=["https://ultralytics.com/images/bus.jpg", "https://ultralytics.com/images/bus.jpg"],
limit=10,
)
print(similar.head())绘制相似图像#
你还可以使用 plot_similar 方法绘制相似图像。此方法接受与 get_similar 相同的参数,并以网格形式绘制相似图像。
from ultralytics import Explorer
# 创建一个 Explorer 对象
exp = Explorer(data="coco128.yaml", model="yolo11n.pt")
exp.create_embeddings_table()
plt = exp.plot_similar(img="https://ultralytics.com/images/bus.jpg", limit=10)
plt.show()2. Ask AI(自然语言查询)#
此功能允许你使用自然语言筛选数据集,而无需编写 SQL。AI 驱动的查询生成器会将你的提示转换为查询并返回匹配结果。例如,你可以询问:“显示 100 张恰好有 1 个人和 2 只狗的图像。也可以有其他对象”,它会生成查询并显示结果。 注意:此功能使用 LLMs,因此结果具有概率性,可能不准确。
from ultralytics.data.explorer import plot_query_result
from ultralytics import Explorer
# 创建一个 Explorer 对象
exp = Explorer(data="coco128.yaml", model="yolo11n.pt")
exp.create_embeddings_table()
df = exp.ask_ai("show me 100 images with exactly one person and 2 dogs. There can be other objects too")
print(df.head())
# 绘制结果
plt = plot_query_result(df)
plt.show()3. SQL 查询#
你可以使用 sql_query 方法对数据集运行 SQL 查询。此方法接受 SQL 查询作为输入,并返回包含结果的 pandas DataFrame。
from ultralytics import Explorer
# 创建一个 Explorer 对象
exp = Explorer(data="coco128.yaml", model="yolo11n.pt")
exp.create_embeddings_table()
df = exp.sql_query("WHERE labels LIKE '%person%' AND labels LIKE '%dog%'")
print(df.head())绘制 SQL 查询结果#
你还可以使用 plot_sql_query 方法绘制 SQL 查询结果。此方法接受与 sql_query 相同的参数,并以网格形式绘制结果。
from ultralytics import Explorer
# 创建一个 Explorer 对象
exp = Explorer(data="coco128.yaml", model="yolo11n.pt")
exp.create_embeddings_table()
# 绘制 SQL 查询
exp.plot_sql_query("WHERE labels LIKE '%person%' AND labels LIKE '%dog%' LIMIT 10")4. 使用嵌入向量表#
你也可以直接使用嵌入向量表。创建嵌入向量表后,可以通过 Explorer.table 访问它。
Explorer 在内部使用 LanceDB 表。你可以使用 Explorer.table 对象直接访问此表并运行原始查询、下推前置和后置筛选条件等。
from ultralytics import Explorer
exp = Explorer()
exp.create_embeddings_table()
table = exp.table以下是一些可以对该表执行的操作示例:
获取原始嵌入向量#
from ultralytics import Explorer
exp = Explorer()
exp.create_embeddings_table()
table = exp.table
embeddings = table.to_pandas()["vector"]
print(embeddings)使用前置和后置筛选条件进行高级查询#
from ultralytics import Explorer
exp = Explorer(model="yolo11n.pt")
exp.create_embeddings_table()
table = exp.table
# 虚拟嵌入向量
embedding = [i for i in range(256)]
rs = table.search(embedding).metric("cosine").where("").limit(10)创建向量索引#
使用大型数据集时,你还可以创建专用向量索引以加快查询速度。你可以对 LanceDB 表使用 create_index 方法来创建索引。
table.create_index(num_partitions=..., num_sub_vectors=...)5. 嵌入向量的应用#
你可以使用嵌入向量表开展各种探索性分析。以下是一些示例:
相似度指数#
Explorer 提供了 similarity_index 操作:
- 此操作会尝试估算每个数据点与数据集中其余数据点的相似程度。
- 它会统计在生成的嵌入空间中,距离当前图像比
max_dist更近的图像嵌入向量数量,每次将top_k张相似图像纳入计算。
它会返回一个 pandas DataFrame,其中包含以下列:
idx:图像在数据集中的索引im_file:图像文件路径count:数据集中距离当前图像比max_dist更近的图像数量sim_im_files:count张相似图像的路径列表
对于给定的数据集、模型、max_dist 和 top_k,生成相似性索引后会重复使用。如果数据集已更改,或者你需要重新生成相似性索引,请传递 force=True。
from ultralytics import Explorer
exp = Explorer()
exp.create_embeddings_table()
sim_idx = exp.similarity_index()你可以使用相似性索引构建自定义条件来筛选数据集。例如,以下代码会列出数据集中相似图像超过 30 张的图像,这些图像很适合用于近重复项去重:
import numpy as np
sim_count = np.array(sim_idx["count"])
sim_idx["im_file"][sim_count > 30]可视化嵌入空间#
你也可以使用自己选择的绘图工具可视化嵌入空间。下面是一个简单示例:使用 PCA 降维嵌入,并用 Matplotlib 绘图:
import matplotlib.pyplot as plt
from sklearn.decomposition import PCA
# 使用 PCA 将维度降至 3 个分量,以便进行 3D 可视化
pca = PCA(n_components=3)
reduced_data = pca.fit_transform(embeddings)
# 使用 Matplotlib Axes3D 创建 3D 散点图
fig = plt.figure(figsize=(8, 6))
ax = fig.add_subplot(111, projection="3d")
# 散点图
ax.scatter(reduced_data[:, 0], reduced_data[:, 1], reduced_data[:, 2], alpha=0.5)
ax.set_title("3D Scatter Plot of Reduced 256-Dimensional Data (PCA)")
ax.set_xlabel("Component 1")
ax.set_ylabel("Component 2")
ax.set_zlabel("Component 3")
plt.show()使用 Explorer API 开始创建自己的计算机视觉数据集探索报告。如需获取灵感,请参阅 VOC 探索示例。
使用 Ultralytics Explorer 构建的应用#
Explorer GUI 基于 Explorer API 构建。
常见问题#
Ultralytics Explorer API 专为数据集探索而设计。它支持通过 SQL 查询、向量相似性搜索和语义搜索来筛选和搜索数据集,并可扩展到大型数据集,用于基于 Ultralytics 模型的计算机视觉任务。
要安装 Ultralytics Explorer API 及其依赖项,请使用以下命令:
pip install "ultralytics[explorer]==8.3.11"这会自动安装 Explorer API 功能所需的所有外部库。如需了解更多设置详情,请参阅文档中的安装部分。
你可以使用 Ultralytics Explorer API 创建嵌入表,并查询相似图像,从而执行相似度搜索。以下是一个基本示例:
from ultralytics import Explorer # 创建一个 Explorer 对象 explorer = Explorer(data="coco128.yaml", model="yolo11n.pt") explorer.create_embeddings_table() # 搜索与给定图像相似的图像 similar_images_df = explorer.get_similar(img="path/to/image.jpg") print(similar_images_df.head())如需了解更多详情,请参阅相似度搜索部分。
Ultralytics Explorer 在底层使用 LanceDB,后者提供可扩展的磁盘嵌入表。这样,你就可以为 COCO 等大型数据集创建并重复使用嵌入,而不必担心内存耗尽。这些表只需创建一次,就可以重复使用,从而提升数据处理效率。
Ask AI 功能支持用户使用自然语言查询筛选数据集。该功能借助 LLM 将这些查询转换为 SQL 查询。示例如下:
from ultralytics import Explorer # 创建一个 Explorer 对象 explorer = Explorer(data="coco128.yaml", model="yolo11n.pt") explorer.create_embeddings_table() # 使用自然语言查询 query_result = explorer.ask_ai("show me 100 images with exactly one person and 2 dogs. There can be other objects too") print(query_result.head())更多示例请参阅 Ask AI 部分。
评论2
提出问题、分享结果,或帮助其他人。
Jie LiuI have downloaded
ultralytics-8.3.118 github zip packageand install it withpip install -e .. Then I usepip install ultralytics[explorer]to install explorer, but I got a error as followedRequirement already satisfied: ultralytics[explorer] in /data/miniconda3/envs/app/lib/python3.10/site-packages (8.3.118) WARNING: ultralytics 8.3.118 does not provide the extra 'explorer' Requirement already satisfied: numpy<=2.1.1,>=1.23.0 in /data/miniconda3/envs/app/lib/python3.10/site-packages (from ultralytics[explorer]) (1.26.4) Requirement already satisfied: matplotlib>=3.3.0 in /data/miniconda3/envs/app/lib/python3.10/site-packages (from ultralytics[explorer]) (3.10.1) Requirement already satisfied: opencv-python>=4.6.0 in /data/miniconda3/envs/app/lib/python3.10/site-packages (from ultralytics[explorer]) (4.11.0.86) Requirement already satisfied: pillow>=7.1.2 in /data/miniconda3/envs/app/lib/python3.10/site-packages (from ultralytics[explorer]) (11.2.1) Requirement already satisfied: pyyaml>=5.3.1 in /data/miniconda3/envs/app/lib/python3.10/site-packages (from ultralytics[explorer]) (6.0.2) Requirement already satisfied: requests>=2.23.0 in /data/miniconda3/envs/app/lib/python3.10/site-packages (from ultralytics[explorer]) (2.32.3) Requirement already satisfied: scipy>=1.4.1 in /data/miniconda3/envs/app/lib/python3.10/site-packages (from ultralytics[explorer]) (1.15.2) Requirement already satisfied: torch>=1.8.0 in /data/miniconda3/envs/app/lib/python3.10/site-packages (from ultralytics[explorer]) (2.3.1) Requirement already satisfied: torchvision>=0.9.0 in /data/miniconda3/envs/app/lib/python3.10/site-packages (from ultralytics[explorer]) (0.18.1) Requirement already satisfied: tqdm>=4.64.0 in /data/miniconda3/envs/app/lib/python3.10/site-packages (from ultralytics[explorer]) (4.67.1) Requirement already satisfied: psutil in /data/miniconda3/envs/app/lib/python3.10/site-packages (from ultralytics[explorer]) (7.0.0) Requirement already satisfied: py-cpuinfo in /data/miniconda3/envs/app/lib/python3.10/site-packages (from ultralytics[explorer]) (9.0.0) Requirement already satisfied: pandas>=1.1.4 in /data/miniconda3/envs/app/lib/python3.10/site-packages (from ultralytics[explorer]) (2.2.3) Requirement already satisfied: seaborn>=0.11.0 in /data/miniconda3/envs/app/lib/python3.10/site-packages (from ultralytics[explorer]) (0.13.2) Requirement already satisfied: ultralytics-thop>=2.0.0 in /data/miniconda3/envs/app/lib/python3.10/site-packages (from ultralytics[explorer]) (2.0.14) Requirement already satisfied: contourpy>=1.0.1 in /data/miniconda3/envs/app/lib/python3.10/site-packages (from matplotlib>=3.3.0->ultralytics[explorer]) (1.3.2) Requirement already satisfied: cycler>=0.10 in /data/miniconda3/envs/app/lib/python3.10/site-packages (from matplotlib>=3.3.0->ultralytics[explorer]) (0.12.1) Requirement already satisfied: fonttools>=4.22.0 in /data/miniconda3/envs/app/lib/python3.10/site-packages (from matplotlib>=3.3.0->ultralytics[explorer]) (4.57.0) Requirement already satisfied: kiwisolver>=1.3.1 in /data/miniconda3/envs/app/lib/python3.10/site-packages (from matplotlib>=3.3.0->ultralytics[explorer]) (1.4.8) Requirement already satisfied: packaging>=20.0 in /data/miniconda3/envs/app/lib/python3.10/site-packages (from matplotlib>=3.3.0->ultralytics[explorer]) (24.2) Requirement already satisfied: pyparsing>=2.3.1 in /data/miniconda3/envs/app/lib/python3.10/site-packages (from matplotlib>=3.3.0->ultralytics[explorer]) (3.2.3) Requirement already satisfied: python-dateutil>=2.7 in /data/miniconda3/envs/app/lib/python3.10/site-packages (from matplotlib>=3.3.0->ultralytics[explorer]) (2.9.0.post0) Requirement already satisfied: pytz>=2020.1 in /data/miniconda3/envs/app/lib/python3.10/site-packages (from pandas>=1.1.4->ultralytics[explorer]) (2022.7.1) Requirement already satisfied: tzdata>=2022.7 in /data/miniconda3/envs/app/lib/python3.10/site-packages (from pandas>=1.1.4->ultralytics[explorer]) (2025.2) Requirement already satisfied: charset-normalizer<4,>=2 in /data/miniconda3/envs/app/lib/python3.10/site-packages (from requests>=2.23.0->ultralytics[explorer]) (3.4.1) Requirement already satisfied: idna<4,>=2.5 in /data/miniconda3/envs/app/lib/python3.10/site-packages (from requests>=2.23.0->ultralytics[explorer]) (3.10) Requirement already satisfied: urllib3<3,>=1.21.1 in /data/miniconda3/envs/app/lib/python3.10/site-packages (from requests>=2.23.0->ultralytics[explorer]) (1.26.20) Requirement already satisfied: certifi>=2017.4.17 in /data/miniconda3/envs/app/lib/python3.10/site-packages (from requests>=2.23.0->ultralytics[explorer]) (2025.4.26) Requirement already satisfied: filelock in /data/miniconda3/envs/app/lib/python3.10/site-packages (from torch>=1.8.0->ultralytics[explorer]) (3.18.0) Requirement already satisfied: typing-extensions>=4.8.0 in /data/miniconda3/envs/app/lib/python3.10/site-packages (from torch>=1.8.0->ultralytics[explorer]) (4.13.2) Requirement already satisfied: sympy in /data/miniconda3/envs/app/lib/python3.10/site-packages (from torch>=1.8.0->ultralytics[explorer]) (1.13.3) Requirement already satisfied: networkx in /data/miniconda3/envs/app/lib/python3.10/site-packages (from torch>=1.8.0->ultralytics[explorer]) (3.4.2) Requirement already satisfied: jinja2 in /data/miniconda3/envs/app/lib/python3.10/site-packages (from torch>=1.8.0->ultralytics[explorer]) (3.1.6) Requirement already satisfied: fsspec in /data/miniconda3/envs/app/lib/python3.10/site-packages (from torch>=1.8.0->ultralytics[explorer]) (2025.3.2) Requirement already satisfied: nvidia-cuda-nvrtc-cu12==12.1.105 in /data/miniconda3/envs/app/lib/python3.10/site-packages (from torch>=1.8.0->ultralytics[explorer]) (12.1.105) Requirement already satisfied: nvidia-cuda-runtime-cu12==12.1.105 in /data/miniconda3/envs/app/lib/python3.10/site-packages (from torch>=1.8.0->ultralytics[explorer]) (12.1.105) Requirement already satisfied: nvidia-cuda-cupti-cu12==12.1.105 in /data/miniconda3/envs/app/lib/python3.10/site-packages (from torch>=1.8.0->ultralytics[explorer]) (12.1.105) Requirement already satisfied: nvidia-cudnn-cu12==8.9.2.26 in /data/miniconda3/envs/app/lib/python3.10/site-packages (from torch>=1.8.0->ultralytics[explorer]) (8.9.2.26) Requirement already satisfied: nvidia-cublas-cu12==12.1.3.1 in /data/miniconda3/envs/app/lib/python3.10/site-packages (from torch>=1.8.0->ultralytics[explorer]) (12.1.3.1) Requirement already satisfied: nvidia-cufft-cu12==11.0.2.54 in /data/miniconda3/envs/app/lib/python3.10/site-packages (from torch>=1.8.0->ultralytics[explorer]) (11.0.2.54) Requirement already satisfied: nvidia-curand-cu12==10.3.2.106 in /data/miniconda3/envs/app/lib/python3.10/site-packages (from torch>=1.8.0->ultralytics[explorer]) (10.3.2.106) Requirement already satisfied: nvidia-cusolver-cu12==11.4.5.107 in /data/miniconda3/envs/app/lib/python3.10/site-packages (from torch>=1.8.0->ultralytics[explorer]) (11.4.5.107) Requirement already satisfied: nvidia-cusparse-cu12==12.1.0.106 in /data/miniconda3/envs/app/lib/python3.10/site-packages (from torch>=1.8.0->ultralytics[explorer]) (12.1.0.106) Requirement already satisfied: nvidia-nccl-cu12==2.20.5 in /data/miniconda3/envs/app/lib/python3.10/site-packages (from torch>=1.8.0->ultralytics[explorer]) (2.20.5) Requirement already satisfied: nvidia-nvtx-cu12==12.1.105 in /data/miniconda3/envs/app/lib/python3.10/site-packages (from torch>=1.8.0->ultralytics[explorer]) (12.1.105) Requirement already satisfied: triton==2.3.1 in /data/miniconda3/envs/app/lib/python3.10/site-packages (from torch>=1.8.0->ultralytics[explorer]) (2.3.1) Requirement already satisfied: nvidia-nvjitlink-cu12 in /data/miniconda3/envs/app/lib/python3.10/site-packages (from nvidia-cusolver-cu12==11.4.5.107->torch>=1.8.0->ultralytics[explorer]) (12.8.93) Requirement already satisfied: six>=1.5 in /data/miniconda3/envs/app/lib/python3.10/site-packages (from python-dateutil>=2.7->matplotlib>=3.3.0->ultralytics[explorer]) (1.17.0) Requirement already satisfied: MarkupSafe>=2.0 in /data/miniconda3/envs/app/lib/python3.10/site-packages (from jinja2->torch>=1.8.0->ultralytics[explorer]) (3.0.2) Requirement already satisfied: mpmath<1.4,>=1.1.0 in /data/miniconda3/envs/app/lib/python3.10/site-packages (from sympy->torch>=1.8.0->ultralytics[explorer]) (1.3.0)when run code
from ultralytics import Explorer # Create an Explorer object explorer = Explorer(data="coco128.yaml", model="yolo11n.pt") # Create embeddings for your dataset explorer.create_embeddings_table() # Search for similar images to a given image/images dataframe = explorer.get_similar(img="path/to/image.jpg") # Or search for similar images to a given index/indices dataframe = explorer.get_similar(idx=0)The error has arisen
Traceback (most recent call last): File "/data/liujie/data/ultralytics-8.3.13/tutorial/debug/explorer.py", line 1, in <module> from ultralytics import Explorer ImportError: cannot import name 'Explorer' from 'ultralytics' (/data/liujie/data/ultralytics-8.3.118/ultralytics/__init__.py)please help me to install and use explorer
Glenn JocherUltralytics(已编辑)Hi @keeper-jie,
The issue you're experiencing is because Ultralytics Explorer has been removed since version 8.3.12, and you're using version 8.3.118. That's why you're seeing the warning that the 'explorer' extra is not provided and why you can't import the Explorer class.
As noted in our documentation, we've moved this functionality to Ultralytics Platform, our platform that offers similar and enhanced capabilities for exploring, visualizing, and managing your datasets without writing code.
If you specifically need the Explorer API functionality in your code, you would need to downgrade to a version prior to 8.3.12. However, we recommend trying Ultralytics Platform instead, as it provides an improved experience with all the dataset exploration features you're looking for.
Let me know if you need any further assistance with Ultralytics Platform or if you have any other questions.