跳至内容

参考资料 hub_sdk/modules/datasets.py

备注

该文件可在https://github.com/ultralytics/hub-sdk/blob/main/hub_sdk/modules/datasets.py 上获取。如果您发现问题,请通过提交 Pull Request🛠️ 帮助修复。谢谢🙏!



hub_sdk.modules.datasets.Datasets

垒球 CRUDClient

一个代表客户端的类,用于通过 CRUD 操作与数据集交互。该类扩展了 CRUDClient 类,并提供了用于处理数据集的特定方法。

属性

名称 类型 说明
hub_client DatasetUpload

DatasetUpload 的实例,用于与模型上传交互。

id (str, None)

数据集的唯一标识符(如果有)。

data dict

用于存储数据集数据的字典。

备注

id "属性在初始化过程中设置,可用于唯一标识数据集。 data "属性用于存储从应用程序接口获取的数据集数据。

源代码 hub_sdk/modules/datasets.py
class Datasets(CRUDClient):
    """
    A class representing a client for interacting with Datasets through CRUD operations. This class extends the
    CRUDClient class and provides specific methods for working with Datasets.

    Attributes:
        hub_client (DatasetUpload): An instance of DatasetUpload used for interacting with model uploads.
        id (str, None): The unique identifier of the dataset, if available.
        data (dict): A dictionary to store dataset data.

    Note:
        The 'id' attribute is set during initialization and can be used to uniquely identify a dataset.
        The 'data' attribute is used to store dataset data fetched from the API.
    """

    def __init__(self, dataset_id: Optional[str] = None, headers: Optional[Dict[str, Any]] = None):
        """
        Initialize a Datasets client.

        Args:
            dataset_id (str): Unique id of the dataset.
            headers (dict, optional): Headers to include in HTTP requests.
        """
        super().__init__("datasets", "dataset", headers)
        self.hub_client = DatasetUpload(headers)
        self.id = dataset_id
        self.data = {}
        if dataset_id:
            self.get_data()

    def get_data(self) -> None:
        """
        Retrieves data for the current dataset instance.

        If a valid dataset ID has been set, it sends a request to fetch the dataset data and stores it in the instance.
        If no dataset ID has been set, it logs an error message.

        Returns:
            (None): The method does not return a value.
        """
        if not self.id:
            self.logger.error("No dataset id has been set. Update the dataset id or create a dataset.")
            return

        try:
            response = super().read(self.id)

            if response is None:
                self.logger.error(f"Received no response from the server for dataset ID: {self.id}")
                return

            # Check if the response has a .json() method (it should if it's a response object)
            if not hasattr(response, "json"):
                self.logger.error(f"Invalid response object received for dataset ID: {self.id}")
                return

            resp_data = response.json()
            if resp_data is None:
                self.logger.error(f"No data received in the response for dataset ID: {self.id}")
                return

            self.data = resp_data.get("data", {})
            self.logger.debug(f"Dataset data retrieved for ID: {self.id}")

        except Exception as e:
            self.logger.error(f"An error occurred while retrieving data for dataset ID: {self.id}, {e}")

    def create_dataset(self, dataset_data: dict) -> None:
        """
        Creates a new dataset with the provided data and sets the dataset ID for the current instance.

        Args:
            dataset_data (dict): A dictionary containing the data for creating the dataset.

        Returns:
            (None): The method does not return a value.
        """
        resp = super().create(dataset_data).json()
        self.id = resp.get("data", {}).get("id")
        self.get_data()

    def delete(self, hard: bool = False) -> Optional[Response]:
        """
        Delete the dataset resource represented by this instance.

        Args:
            hard (bool, optional): If True, perform a hard delete.

        Note:
            The 'hard' parameter determines whether to perform a soft delete (default) or a hard delete.
            In a soft delete, the dataset might be marked as deleted but retained in the system.
            In a hard delete, the dataset is permanently removed from the system.

        Returns:
            (Optional[Response]): Response object from the delete request, or None if delete fails.
        """
        return super().delete(self.id, hard)

    def update(self, data: dict) -> Optional[Response]:
        """
        Update the dataset resource represented by this instance.

        Args:
            data (dict): The updated data for the dataset resource.

        Returns:
            (Optional[Response]): Response object from the update request, or None if update fails.
        """
        return super().update(self.id, data)

    def upload_dataset(self, file: str = None) -> Optional[Response]:
        """
        Uploads a dataset file to the hub.

        Args:
            file (str, optional): The path to the dataset file to upload.

        Returns:
            (Optional[Response]): Response object from the upload request, or None if upload fails.
        """
        return self.hub_client.upload_dataset(self.id, file)

    def get_download_link(self) -> Optional[str]:
        """
        Get dataset download link.

        Returns:
            (Optional[str]): Return download link or None if the link is not available.
        """
        return self.data.get("url")

__init__(dataset_id=None, headers=None)

初始化 Datasets 客户端。

参数

名称 类型 说明 默认值
dataset_id str

数据集的唯一 ID。

None
headers dict

要包含在 HTTP 请求中的标头。

None
源代码 hub_sdk/modules/datasets.py
def __init__(self, dataset_id: Optional[str] = None, headers: Optional[Dict[str, Any]] = None):
    """
    Initialize a Datasets client.

    Args:
        dataset_id (str): Unique id of the dataset.
        headers (dict, optional): Headers to include in HTTP requests.
    """
    super().__init__("datasets", "dataset", headers)
    self.hub_client = DatasetUpload(headers)
    self.id = dataset_id
    self.data = {}
    if dataset_id:
        self.get_data()

create_dataset(dataset_data)

使用提供的数据创建新数据集,并为当前实例设置数据集 ID。

参数

名称 类型 说明 默认值
dataset_data dict

包含用于创建数据集的数据的字典。

所需

返回:

类型 说明
None

该方法不返回值。

源代码 hub_sdk/modules/datasets.py
def create_dataset(self, dataset_data: dict) -> None:
    """
    Creates a new dataset with the provided data and sets the dataset ID for the current instance.

    Args:
        dataset_data (dict): A dictionary containing the data for creating the dataset.

    Returns:
        (None): The method does not return a value.
    """
    resp = super().create(dataset_data).json()
    self.id = resp.get("data", {}).get("id")
    self.get_data()

delete(hard=False)

删除该实例所代表的数据集资源。

参数

名称 类型 说明 默认值
hard bool

如果为 True,则执行硬删除。

False
备注

硬 "参数决定执行软删除(默认)还是硬删除。 在软删除中,数据集可能被标记为已删除,但仍保留在系统中。 在硬删除中,数据集将从系统中永久删除。

返回:

类型 说明
Optional[Response]

删除请求的响应对象;如果删除失败,则为 "无"。

源代码 hub_sdk/modules/datasets.py
def delete(self, hard: bool = False) -> Optional[Response]:
    """
    Delete the dataset resource represented by this instance.

    Args:
        hard (bool, optional): If True, perform a hard delete.

    Note:
        The 'hard' parameter determines whether to perform a soft delete (default) or a hard delete.
        In a soft delete, the dataset might be marked as deleted but retained in the system.
        In a hard delete, the dataset is permanently removed from the system.

    Returns:
        (Optional[Response]): Response object from the delete request, or None if delete fails.
    """
    return super().delete(self.id, hard)

get_data()

读取当前数据集实例的数据。

如果设置了有效的数据集 ID,它就会发送请求获取数据集数据并将其存储到实例中。 如果没有设置数据集 ID,则会记录一条错误信息。

返回:

类型 说明
None

该方法不返回值。

源代码 hub_sdk/modules/datasets.py
def get_data(self) -> None:
    """
    Retrieves data for the current dataset instance.

    If a valid dataset ID has been set, it sends a request to fetch the dataset data and stores it in the instance.
    If no dataset ID has been set, it logs an error message.

    Returns:
        (None): The method does not return a value.
    """
    if not self.id:
        self.logger.error("No dataset id has been set. Update the dataset id or create a dataset.")
        return

    try:
        response = super().read(self.id)

        if response is None:
            self.logger.error(f"Received no response from the server for dataset ID: {self.id}")
            return

        # Check if the response has a .json() method (it should if it's a response object)
        if not hasattr(response, "json"):
            self.logger.error(f"Invalid response object received for dataset ID: {self.id}")
            return

        resp_data = response.json()
        if resp_data is None:
            self.logger.error(f"No data received in the response for dataset ID: {self.id}")
            return

        self.data = resp_data.get("data", {})
        self.logger.debug(f"Dataset data retrieved for ID: {self.id}")

    except Exception as e:
        self.logger.error(f"An error occurred while retrieving data for dataset ID: {self.id}, {e}")

获取数据集下载链接。

返回:

类型 说明
Optional[str]

返回下载链接,如果链接不可用,则返回无。

源代码 hub_sdk/modules/datasets.py
def get_download_link(self) -> Optional[str]:
    """
    Get dataset download link.

    Returns:
        (Optional[str]): Return download link or None if the link is not available.
    """
    return self.data.get("url")

update(data)

更新该实例所代表的数据集资源。

参数

名称 类型 说明 默认值
data dict

数据集资源的更新数据。

所需

返回:

类型 说明
Optional[Response]

更新请求的响应对象;如果更新失败,则为 "无"。

源代码 hub_sdk/modules/datasets.py
def update(self, data: dict) -> Optional[Response]:
    """
    Update the dataset resource represented by this instance.

    Args:
        data (dict): The updated data for the dataset resource.

    Returns:
        (Optional[Response]): Response object from the update request, or None if update fails.
    """
    return super().update(self.id, data)

upload_dataset(file=None)

将数据集文件上传到中心。

参数

名称 类型 说明 默认值
file str

要上传的数据集文件的路径。

None

返回:

类型 说明
Optional[Response]

上传请求的响应对象;如果上传失败,则为 "无"。

源代码 hub_sdk/modules/datasets.py
def upload_dataset(self, file: str = None) -> Optional[Response]:
    """
    Uploads a dataset file to the hub.

    Args:
        file (str, optional): The path to the dataset file to upload.

    Returns:
        (Optional[Response]): Response object from the upload request, or None if upload fails.
    """
    return self.hub_client.upload_dataset(self.id, file)



hub_sdk.modules.datasets.DatasetList

垒球 PaginatedList

A class for managing a paginated list of datasets from the Ultralytics Hub API.

源代码 hub_sdk/modules/datasets.py
class DatasetList(PaginatedList):
    """A class for managing a paginated list of datasets from the Ultralytics Hub API."""

    def __init__(self, page_size=None, public=None, headers=None):
        """
        Initialize a Dataset instance.

        Args:
            page_size (int, optional): The number of items to request per page.
            public (bool, optional): Whether the items should be publicly accessible.
            headers (dict, optional): Headers to be included in API requests.
        """
        base_endpoint = "datasets"
        super().__init__(base_endpoint, "dataset", page_size, public, headers)

__init__(page_size=None, public=None, headers=None)

初始化数据集实例。

参数

名称 类型 说明 默认值
page_size int

每页要申请的项目数。

None
public bool

项目是否应向公众开放。

None
headers dict

应包含在 API 请求中的标头。

None
源代码 hub_sdk/modules/datasets.py
def __init__(self, page_size=None, public=None, headers=None):
    """
    Initialize a Dataset instance.

    Args:
        page_size (int, optional): The number of items to request per page.
        public (bool, optional): Whether the items should be publicly accessible.
        headers (dict, optional): Headers to be included in API requests.
    """
    base_endpoint = "datasets"
    super().__init__(base_endpoint, "dataset", page_size, public, headers)