AMD Xilinx Deployment for Ultralytics YOLO with Vitis AI#

AMD Xilinx devices power many of the world's industrial cameras, automotive vision systems, robots, drones and medical imaging products. They combine Arm processors with programmable logic and, on newer devices, dedicated AI Engines, so a single chip can capture video, preprocess it, run object detection and act on the result with low, predictable inference latency.

This guide covers what each AMD Xilinx device family is, how AI runs on them, which Ultralytics YOLO operators each accelerator supports, and the step-by-step workflow for deploying YOLO models on Zynq UltraScale+, Kria and Versal hardware.

What is AMD Xilinx?#

Xilinx invented the field-programmable gate array (FPGA) in the 1980s and became a leading supplier of adaptive SoCs and FPGAs. AMD completed its acquisition of Xilinx in February 2022, and the product lines are now sold under the AMD brand as AMD Zynq, AMD Kria, AMD Versal and AMD Vitis.

Key Terms and Concepts#

AMD Xilinx deployment uses its own vocabulary. The table below explains every term used in this guide.

AMD Xilinx Device Families for Edge AI#

AMD Xilinx devices for edge AI fall into three families. Zynq and Kria use the DPU in programmable logic, while the Versal AI Edge devices covered here use the NPU on their AI Engines.

Zynq UltraScale+ MPSoC#

Each Zynq UltraScale+ chip pairs an Arm processing system, with dual-core (CG) or quad-core (EG and EV) Cortex-A53 cores and real-time Cortex-R5F cores, with FPGA logic. EV devices add a hardened H.264/H.265 video codec. To run neural networks, designers build a DPU into the logic next to their camera and video pipelines. On small devices the DPU competes for space with the rest of the design.

Kria System-on-Modules#

The Kria K26 module packages a Zynq UltraScale+ MPSoC, memory and power on a production-ready module, so you avoid designing the processor, memory and power subsystem yourself. The module plugs into a carrier card, either a starter-kit board or your own design. It powers the KV260 Vision AI Starter Kit for smart cameras and the KR260 Robotics Starter Kit for robotics. Because the K26 is built on Zynq UltraScale+, it uses the same DPU flow. The Kria portfolio also includes other modules, so check which processor your module uses before choosing a flow.

Versal Adaptive SoCs#

Versal is AMD's adaptive SoC family. Its AI Edge and AI Core series add hardened AI Engines next to the Arm cores and programmable logic, while some other Versal series have no AI Engines. AMD's NPU IP runs on AI Engines and programmable logic together, and Vitis AI targets the AIE-ML parts of the AI Edge series, such as the VE2302 and VE2802. The Versal AI Edge Series (VEK280 evaluation kit) and the Versal AI Edge Series Gen 2 (VEK385 evaluation kit) are AMD's current targets for edge AI and the focus of current Vitis AI releases.

How AI Runs on AMD Xilinx Devices: DPU vs NPU#

Most AMD Xilinx AI deployments follow the same pattern. The accelerator runs the layers it supports, the Arm CPU runs preprocessing, post-processing and any layers the accelerator cannot execute, and a runtime on the board coordinates the two.

graph LR
    A[Camera / video input]:::start --> B[Arm CPU<br>Linux, preprocessing,<br>post-processing]:::proc
    B <--> C[AI accelerator<br>DPU in programmable logic<br>or NPU on AI Engines + PL]:::out
    B --> D[Application<br>alerts, control, display]:::start

    classDef start fill:#4CAF50,color:#fff
    classDef proc fill:#2196F3,color:#fff
    classDef out fill:#9C27B0,color:#fff

AMD has shipped two generations of accelerator, each with its own toolchain and compiled model file. This guide follows Vitis AI 3.5 for the DPU and Vitis AI 6.3 for the NPU; check AMD's current documentation for later releases.

Which Vitis AI Flow Do I Need?#

Pick your flow from the device on your board:

graph TD
    A[Start: which AMD device<br>is on your board?]:::start --> B{Device family?}:::decide
    B -->|Zynq UltraScale+ MPSoC<br>or Kria K26| C[DPU flow<br>Vitis AI 3.5]:::proc
    B -->|Versal AI Edge<br>VEK280| D[NPU snapshot flow<br>Vitis AI 6.3]:::proc
    B -->|Versal AI Edge Gen 2<br>VEK385| E[NPU Quark flow<br>Vitis AI 6.3]:::proc
    C --> F[Train YOLO with Hard-Swish<br>then compile to .xmodel]:::out
    D --> G[Run your model on calibration<br>images to capture a snapshot]:::out
    E --> H[Quantize ONNX with Quark<br>then compile to .rai]:::out

    classDef start fill:#4CAF50,color:#fff
    classDef proc fill:#2196F3,color:#fff
    classDef decide fill:#FF9800,color:#fff
    classDef out fill:#9C27B0,color:#fff

YOLO Model Compatibility and Supported Operators#

An accelerator only speeds up the operators it implements in hardware. When a model contains an unsupported operator, the compiler usually sends that part of the network to the Arm CPU, and each round trip between the accelerator and the CPU adds latency. Some operators cannot be partitioned: on the Versal AI Edge Gen 2 NPU, AMD lists operators such as NonZero and NonMaxSuppression that can force the entire model onto the CPU. Operator support is the most important factor in how well a YOLO model performs on AMD Xilinx hardware.

graph LR
    subgraph S1 [Stock YOLO26 on the DPU]
        A1[Conv]:::out --> A2[SiLU<br>CPU]:::error --> A3[Conv]:::out --> A4[SiLU<br>CPU]:::error --> A5[...]:::proc
    end
    subgraph S2 [Hard-Swish YOLO26 on the DPU]
        B1[Backbone<br>Conv + Hard-Swish<br>DPU]:::out --> B2[C2PSA attention<br>CPU]:::error --> B3[Neck<br>DPU]:::out --> B4[C3k2 attention<br>CPU]:::error --> B5[Detect head<br>DPU]:::out --> B6[Sigmoid and<br>post-processing<br>CPU]:::error
    end

    classDef proc fill:#2196F3,color:#fff
    classDef out fill:#9C27B0,color:#fff
    classDef error fill:#F44336,color:#fff

The table shows where each operator in a YOLO26 model runs. A stock YOLO26n ONNX export contains 87 SiLU activations, each exported as a Sigmoid and a Mul, plus 4 MatMul and 2 Softmax operators from its two attention blocks: the C2PSA block at the end of the backbone (layer 10) and the attention-enabled C3k2 block that produces the P5 output (layer 22).

Sources: AMD UG1414 supported operators, PyTorch operator support, and the Versal AI Edge Gen 2 supported, CPU partition and unsupported operator lists. Support also depends on your DPU configuration and graph patterns, so always check the compiler's partition report.

Make YOLO26 DPU-Ready with Hard-Swish#

The DPU fuses only ReLU, ReLU6, LeakyReLU, Hard-Swish and Hard-Sigmoid into its convolutions. Hard-Swish is a hardware-friendly approximation of SiLU, which makes it the natural replacement. Ultralytics model YAML files accept an activation key that changes the default activation of Conv blocks (Model YAML Configuration Guide).

Copy yolo26.yaml to yolo26-hswish.yaml and add one line under the parameters:

# Parameters
nc: 80 # number of classes
activation: nn.Hardswish() # default Conv activation, DPU-native
end2end: True # whether to use end-to-end mode

Then build the model, transfer the pretrained YOLO26 weights and fine-tune on your dataset:

from ultralytics import YOLO

# Build YOLO26n with Hard-Swish activations; the 'n' in the name selects the nano scale
model = YOLO("yolo26n-hswish.yaml").load("yolo26n.pt")  # transfer pretrained weights

# Fine-tune so the network adapts to Hard-Swish
model.train(data="coco8.yaml", epochs=100, imgsz=640)

Activations have no weights, so all of the pretrained weights transfer. The exported ONNX graph then contains 87 HardSwish operators and no SiLU. Replace coco8.yaml with your own dataset, and compare accuracy against the SiLU model with Val mode before deployment.

Handling Attention Blocks on the DPU#

YOLO26 applies attention at the lowest resolution (a 20×20 grid at 640 input) in two places: the C2PSA block at layer 10 and the attention-enabled C3k2 block at layer 22. YOLO11 has one C2PSA block. On a DPU their MatMul and Softmax operators run on the CPU, which splits the model into alternating DPU and CPU subgraphs. You have three options:

1. Accept the CPU blocks. The compiled .xmodel then contains CPU subgraphs, so run it with AMD's Graph Runner, which executes DPU and CPU subgraphs together when a CPU implementation exists for every operator; otherwise you must implement and register the missing operators. At 20×20 the attention computation is small, but each extra transfer between the DPU and the CPU adds latency, so measure it on your board.

2. Use an attention-free YAML. In your Hard-Swish YAML, replace the C2PSA layer with nn.Identity so the layer indices used by Concat and Detect stay valid, and disable attention in layer 22:

   backbone:
       # ... layers 0-9 unchanged
       - [-1, 1, nn.Identity, []] # 10 C2PSA removed; keeps later layer indices valid
   
   head:
       # ... layers 11-21 unchanged
       - [-1, 1, C3k2, [1024, True, 0.5, False]] # 22 (P5/32-large), attention disabled
       - [[16, 19, 22], 1, Detect, [nc]] # Detect(P3, P4, P5)

   The exported ONNX graph then contains no MatMul or Softmax operators. The attention weights no longer apply (624 of 666 YOLO26n weights transfer), so fine-tune longer and compare accuracy with Val mode.

3. Use an attention-free model such as YOLOv8, which AMD has used in its own DPU examples.

On Versal AI Edge Gen 2, attention operators are listed as NPU-supported, so these changes are usually unnecessary. Confirm placement in the compiler report, because AMD notes that supported operators can still fall back to the CPU because of configuration or memory constraints.

Model Compatibility at a Glance#

Deploy YOLO26 on AMD Xilinx Today#

Until native export is available, deployment follows four steps:

graph LR
    A[1. Train or fine-tune<br>Ultralytics YOLO]:::start --> B{Target?}:::decide
    B -->|Versal NPU| C[2. Export to ONNX<br>model.export]:::proc
    B -->|Zynq or Kria DPU| D[2. Keep the trained<br>PyTorch checkpoint]:::proc
    C --> E[3. Quantize and compile<br>Vitis AI 6.3 Docker]:::proc
    D --> F[3. Quantize and compile<br>Vitis AI 3.5 Docker]:::proc
    E --> G[4. Run on the board<br>VART-ML or ONNX Runtime]:::out
    F --> H[4. Run on the board<br>VART]:::out
    G -.->|accuracy check| A
    H -.->|accuracy check| A

    classDef start fill:#4CAF50,color:#fff
    classDef proc fill:#2196F3,color:#fff
    classDef decide fill:#FF9800,color:#fff
    classDef out fill:#9C27B0,color:#fff

Step 1: Train or Fine-Tune Your Model#

Train on your own data with Train mode or on the Ultralytics Platform. For DPU targets, start from the Hard-Swish YAML. Record a baseline with Val mode so you can measure the accuracy impact of quantization later.

Step 2: Export to ONNX for NPU Targets#

ONNX is the common input to AMD's NPU flows. The DPU flow quantizes the trained PyTorch checkpoint directly in the Vitis AI 3.5 Docker image, so DPU users can skip this step. Export with a fixed batch size of 1 and an opset that AMD supports; AMD's YOLOv8m tutorial for Versal AI Edge Gen 2 uses opset 17.

from ultralytics import YOLO

# Load the model you trained in Step 1
model = YOLO("runs/detect/train/weights/best.pt")

# Export to ONNX with a static shape for the AMD compiler
model.export(format="onnx", opset=17, imgsz=640)  # creates 'best.onnx' next to 'best.pt'

See the ONNX integration and export arguments for all options. With nms unset, run NMS on the CPU after inference; for YOLO26, nms=False selects the NMS-free head instead. Do not embed NMS with nms=True, because AMD lists NonMaxSuppression among the operators that can force the entire model onto the CPU.

Step 3: Quantize and Compile with Vitis AI#

The workflows below use AMD's Docker images on an x86-64 Linux host. You do not need the board for this step. Choose the tab for your device:

Step 4: Run and Validate on the Board#

Prepare the board first. It must run a hardware design and Linux image that contain the accelerator configuration you compiled for, plus the matching Vitis AI runtime. See AMD's setup guides for Zynq UltraScale+ and Kria DPU targets, Versal AI Edge (VEK280) and Versal AI Edge Gen 2 (VEK385).

Then copy the artifacts your runtime needs:

For the NPU flows, the exported ONNX graph already decodes boxes and applies the class-score sigmoid, so the host only interprets the output:

• nms unset: detection models output a (1, 4 + nc, anchors) tensor of xywh boxes and per-class scores. Select the best class per anchor, convert boxes to corners, filter by confidence and run NMS; the Ultralytics non_max_suppression function performs all of these steps.
• nms=False (YOLO26): the model outputs a (1, max_det, 6) tensor of [x1, y1, x2, y2, score, class] rows, which only needs a confidence threshold.

In both cases, rescale boxes from the letterboxed input back to the original image. Only graphs cut before the decode step need box decoding on the host. On the DPU, VART buffers hold fixed-point INT8 values: query each tensor's shape and fix_point scale, quantize inputs and dequantize outputs before applying the steps above. The Graph Runner returns the full graph's outputs, while a DPU-only runner returns intermediate DPU subgraph outputs that your code must finish computing. The layouts above are the ONNX (CPU view) layouts: VART-ML defaults to hardware tensor views whose shape, data type and memory layout can differ, so configure the runner's input and output tensor types as CPU views or convert the hardware format yourself (see AMD's VART-ML architecture overview).

Compare on-device accuracy with the FP32 baseline from Step 1 using your own validation set and the same performance metrics, such as mAP. AMD's published YOLOv8m results on the VEK385 show the accuracy cost of INT8 deployment:

Source: AMD YOLOv8m tutorial for Versal AI Edge Gen 2, which also reports a 10.69 ms average inference time over 100 VART runs at dp_size=1.

Real-World Applications#

AMD Xilinx devices are common wherever vision AI must run in real time, at low power, close to the sensor:

• Industrial and construction safety: Detect people and machines around heavy equipment, and monitor work zones with object detection and object counting.
• Automotive and off-highway vision: Run camera-based perception on qualified automotive-grade parts, supporting autonomous vehicles and driver assistance.
• Smart cameras and video analytics: Combine video capture, encoding and YOLO inference on one chip for security systems and video analytics.
• Machine vision and quality inspection: Pair FPGA-based high-speed image capture with YOLO instance segmentation or classification for inline defect detection.
• Robotics and drones: Use Kria KR260 or Versal modules for pose estimation, oriented object detection and navigation with deterministic latency.

Summary#

AMD Xilinx devices run YOLO models through two accelerator generations. The DPU on Zynq UltraScale+ and Kria uses the frozen Vitis AI 3.5 flow and produces .xmodel files. It needs DPU-native activations such as Hard-Swish, and it runs attention on the CPU. The NPU on Versal AI Edge and Versal AI Edge Gen 2 uses current Vitis AI releases. The Gen 2 NPU supports SiLU and attention operators and runs AMD's YOLOv8m example on the VEK385 almost entirely on the NPU, while operator coverage on the earlier VEK280 NPU depends on the Vitis AI version and precision.

Native Ultralytics export for AMD Xilinx devices is coming soon. Until then, train with Ultralytics, export to ONNX for Versal NPU targets or keep the PyTorch checkpoint for DPU targets, and compile with Vitis AI as described above. For other deployment targets, see the model deployment options guide, deployment best practices, and accelerator integrations such as Hailo, Rockchip RKNN and Axelera.

FAQ#