A Guide to Deploying YOLO11 on Amazon SageMaker Endpoints

Q: How can I test the deployed YOLO11 model on Amazon SageMaker?

To test the deployed YOLO11 model on Amazon SageMaker: For comprehensive testing instructions, visit the testing section.

Deploying advanced computer vision models like Ultralytics' YOLO11 on Amazon SageMaker Endpoints opens up a wide range of possibilities for various machine learning applications. The key to effectively using these models lies in understanding their setup, configuration, and deployment processes. YOLO11 becomes even more powerful when integrated seamlessly with Amazon SageMaker, a robust and scalable machine learning service by AWS.

This guide will take you through the process of deploying YOLO11 PyTorch models on Amazon SageMaker Endpoints step by step. You'll learn the essentials of preparing your AWS environment, configuring the model appropriately, and using tools like AWS CloudFormation and the AWS Cloud Development Kit (CDK) for deployment.

亚马逊 SageMaker

亚马逊 SageMaker 概览

Amazon SageMaker是亚马逊网络服务（AWS）提供的一项机器学习服务，可简化机器学习模型的构建、训练和部署过程。它提供了广泛的工具，用于处理机器学习工作流的各个方面。其中包括调整模型的自动化功能、大规模训练模型的选项以及将模型部署到生产中的直接方法。SageMaker 支持流行的机器学习框架，为各种项目提供所需的灵活性。它的功能还包括数据标记、工作流管理和性能分析。

Deploying YOLO11 on Amazon SageMaker Endpoints

Deploying YOLO11 on Amazon SageMaker lets you use its managed environment for real-time inference and take advantage of features like autoscaling. Take a look at the AWS architecture below.

AWS 架构

步骤 1：设置 AWS 环境

首先，确保具备以下先决条件：

AWS 账户：如果还没有，请注册一个 AWS 账户。
已配置的 IAM 角色：您需要一个具有 Amazon SageMaker、AWS CloudFormation 和 Amazon S3 必要权限的 IAM 角色。此角色应具有允许其访问这些服务的策略。
AWSCLI ：如果尚未安装，请下载并安装 AWS 命令行界面 (CLI) 并将其配置为您的账户详细信息。请按照AWSCLI 的说明进行安装。
AWS CDK：如果尚未安装，请安装 AWS 云开发工具包 (CDK)，该工具包将用于编写部署脚本。请按照AWS CDK 的说明进行安装。
足够的服务配额：确认为 Amazon SageMaker 中的两个独立资源提供了足够的配额：一个用于 ml.m5.4xlarge 用于终端使用，另一个用于 ml.m5.4xlarge 用于笔记本实例的使用。每个配额至少需要一个配额值。如果您当前的配额低于这一要求，请务必申请增加配额。您可以按照以下详细说明申请增加配额 AWS 服务配额文档.

Step 2: Clone the YOLO11 SageMaker Repository

The next step is to clone the specific AWS repository that contains the resources for deploying YOLO11 on SageMaker. This repository, hosted on GitHub, includes the necessary CDK scripts and configuration files.

克隆 GitHub 仓库：在终端中执行以下命令克隆 host-yolov8-on-sagemaker-endpoint 仓库：

git clone https://github.com/aws-samples/host-yolov8-on-sagemaker-endpoint.git

cd host-yolov8-on-sagemaker-endpoint/yolov8-pytorch-cdk

步骤 3：设置 CDK 环境

现在您已经有了必要的代码，请设置您的环境以便使用 AWS CDK 进行部署。

创建Python 虚拟环境：这将隔离Python 环境和依赖项。运行：

python3 -m venv .venv

激活虚拟环境：

source .venv/bin/activate

安装依赖项：安装项目所需的Python 依赖项：

pip3 install -r requirements.txt

升级 AWS CDK 库：确保您拥有最新版本的 AWS CDK 库：

pip install --upgrade aws-cdk-lib

第 4 步：创建 AWS CloudFormation 堆栈

合成 CDK 应用程序：根据 CDK 代码生成 AWS CloudFormation 模板：

cdk synth

引导 CDK 应用程序：为 CDK 部署准备 AWS 环境：

cdk bootstrap

部署堆栈：这将创建必要的 AWS 资源并部署您的模型：

cdk deploy

Step 5: Deploy the YOLO Model

Before diving into the deployment instructions, be sure to check out the range of YOLO11 models offered by Ultralytics. This will help you choose the most appropriate model for your project requirements.

After creating the AWS CloudFormation Stack, the next step is to deploy YOLO11.

打开笔记本实例：转到 AWS 控制台并导航到 Amazon SageMaker 服务。从仪表板中选择 "笔记本实例"，然后找到 CDK 部署脚本创建的笔记本实例。打开笔记本实例，访问 Jupyter 环境。
访问并修改 inference.py：在 Jupyter 中打开 SageMaker 笔记本实例后，找到 inference.py 文件。按下图所示编辑 inference.py 中的 output_fn 函数，并将更改保存到脚本，确保没有语法错误。

import json


def output_fn(prediction_output):
    """Formats model outputs as JSON string, extracting attributes like boxes, masks, keypoints."""
    print("Executing output_fn from inference.py ...")
    infer = {}
    for result in prediction_output:
        if result.boxes is not None:
            infer["boxes"] = result.boxes.numpy().data.tolist()
        if result.masks is not None:
            infer["masks"] = result.masks.numpy().data.tolist()
        if result.keypoints is not None:
            infer["keypoints"] = result.keypoints.numpy().data.tolist()
        if result.obb is not None:
            infer["obb"] = result.obb.numpy().data.tolist()
        if result.probs is not None:
            infer["probs"] = result.probs.numpy().data.tolist()
    return json.dumps(infer)

Deploy the Endpoint Using 1_DeployEndpoint.ipynb: In the Jupyter environment, open the 1_DeployEndpoint.ipynb notebook located in the sm-notebook directory. Follow the instructions in the notebook and run the cells to download the YOLO11 model, package it with the updated inference code, and upload it to an Amazon S3 bucket. The notebook will guide you through creating and deploying a SageMaker endpoint for the YOLO11 model.

步骤 6：测试部署

Now that your YOLO11 model is deployed, it's important to test its performance and functionality.

打开测试笔记本：在同一 Jupyter 环境中，找到并打开同样位于 sm-notebook 目录中的 2_TestEndpoint.ipynb 笔记本。
Run the Test Notebook: Follow the instructions within the notebook to test the deployed SageMaker endpoint. This includes sending an image to the endpoint and running inferences. Then, you'll plot the output to visualize the model's performance and accuracy, as shown below.

Testing Results YOLO11

清理资源：测试笔记本还将指导您完成清理终端和托管模型的过程。这是有效管理成本和资源的重要步骤，尤其是在不打算立即使用已部署模型的情况下。

步骤 7：监测和管理

测试之后，对部署的模型进行持续监控和管理至关重要。

使用 Amazon CloudWatch 进行监控：使用Amazon CloudWatch 定期检查 SageMaker 端点的性能和健康状况。
管理终端：使用 SageMaker 控制台对端点进行持续管理。这包括根据需要扩展、更新或重新部署模型。

By completing these steps, you will have successfully deployed and tested a YOLO11 model on Amazon SageMaker Endpoints. This process not only equips you with practical experience in using AWS services for machine learning deployment but also lays the foundation for deploying other advanced models in the future.

摘要

This guide took you step by step through deploying YOLO11 on Amazon SageMaker Endpoints using AWS CloudFormation and the AWS Cloud Development Kit (CDK). The process includes cloning the necessary GitHub repository, setting up the CDK environment, deploying the model using AWS services, and testing its performance on SageMaker.

有关更多技术细节，请参阅 AWS 机器学习博客上的这篇文章。您还可以查看亚马逊 SageMaker官方文档，了解有关各种特性和功能的更多信息。

Are you interested in learning more about different YOLO11 integrations? Visit the Ultralytics integrations guide page to discover additional tools and capabilities that can enhance your machine-learning projects.

常见问题

How do I deploy the Ultralytics YOLO11 model on Amazon SageMaker Endpoints?

To deploy the Ultralytics YOLO11 model on Amazon SageMaker Endpoints, follow these steps:

设置 AWS 环境：确保您拥有 AWS 账户、具有必要权限的 IAM 角色，并配置了 AWSCLI 。安装 AWS CDK（如果尚未安装）（请参阅AWS CDK 说明）。

Clone the YOLO11 SageMaker Repository:

git clone https://github.com/aws-samples/host-yolov8-on-sagemaker-endpoint.git
cd host-yolov8-on-sagemaker-endpoint/yolov8-pytorch-cdk

设置 CDK 环境:创建Python 虚拟环境、激活、安装依赖项并升级 AWS CDK 库。

python3 -m venv .venv
source .venv/bin/activate
pip3 install -r requirements.txt
pip install --upgrade aws-cdk-lib

使用 AWS CDK 进行部署:合成和部署 CloudFormation 堆栈，启动环境。
```
cdk synth
cdk bootstrap
cdk deploy
```

有关更多详细信息，请查看文档部分。

What are the prerequisites for deploying YOLO11 on Amazon SageMaker?

To deploy YOLO11 on Amazon SageMaker, ensure you have the following prerequisites:

AWS 账户：活跃的 AWS 账户（在此注册）。
IAM 角色：为 SageMaker、CloudFormation 和 Amazon S3 配置具有权限的 IAM 角色。
AWSCLI ：安装并配置 AWS 命令行界面（AWSCLI 安装指南）。
AWS CDK：已安装 AWS 云开发工具包（CDK 安装指南）。
服务配额:足够的配额 ml.m5.4xlarge 终端和笔记本使用的实例 (要求增加配额).

有关详细设置，请参阅本节。

Why should I use Ultralytics YOLO11 on Amazon SageMaker?

Using Ultralytics YOLO11 on Amazon SageMaker offers several advantages:

可扩展性和管理：SageMaker 提供可管理的环境，具有自动扩展等功能，有助于满足实时推理需求。
与 AWS 服务集成：与其他 AWS 服务无缝集成，如用于数据存储的 S3、用于基础设施即代码的 CloudFormation 以及用于监控的 CloudWatch。
易于部署：使用 AWS CDK 脚本简化设置，并精简部署流程。
性能：利用 Amazon SageMaker 的高性能基础设施，高效运行大规模推理任务。

更多关于使用 SageMaker 的优势，请参阅介绍部分。

Can I customize the inference logic for YOLO11 on Amazon SageMaker?

Yes, you can customize the inference logic for YOLO11 on Amazon SageMaker:

修改 inference.py:找到并自定义 output_fn 功能中的 inference.py 文件来定制输出格式。

import json


def output_fn(prediction_output):
    """Formats model outputs as JSON string, extracting attributes like boxes, masks, keypoints."""
    infer = {}
    for result in prediction_output:
        if result.boxes is not None:
            infer["boxes"] = result.boxes.numpy().data.tolist()
        # Add more processing logic if necessary
    return json.dumps(infer)