Empezar

Detección interactiva de objetos: Gradio y Ultralytics YOLO26

Introducción a la detección interactiva de objetos#

Esta interfaz de Gradio ofrece una forma sencilla e interactiva de realizar detección de objetos con el modelo Ultralytics YOLO26. Puedes subir imágenes y ajustar parámetros como el umbral de confianza y el umbral de intersección sobre unión (IoU) para obtener resultados de detección en tiempo real.



Ver: Implementa Ultralytics YOLO26 en un navegador con Gradio | Tutorial completo | IA para visión 🌐

¿Por qué usar Gradio para la detección de objetos?#

  • Interfaz fácil de usar: Gradio ofrece una plataforma sencilla para subir imágenes y visualizar los resultados de detección, sin necesidad de programar.
  • Ajustes en tiempo real: Puedes ajustar sobre la marcha parámetros como los umbrales de confianza e IoU y obtener comentarios inmediatos para optimizar los resultados de detección.
  • Amplia accesibilidad: Cualquier persona puede acceder a la interfaz web de Gradio, lo que la hace útil para demostraciones, actividades educativas y experimentos rápidos.

Gradio YOLO detection interface

Cómo instalar Gradio#

pip install gradio

Cómo usar la interfaz#

  1. Sube una imagen: Haz clic en «Subir imagen» para elegir un archivo de imagen para la detección de objetos.
  2. Ajusta los parámetros:
    • Umbral de confianza: Deslizador para establecer el nivel mínimo de confianza para detectar objetos.
    • Umbral de IoU: Control deslizante para ajustar el umbral de IoU que utiliza la supresión no máxima para eliminar cuadros superpuestos.
  3. Consulta los resultados: Se mostrará la imagen procesada con los objetos detectados y sus etiquetas.

Ejemplos de uso#

  • Imagen de ejemplo 1: Detección de autobuses con los umbrales predeterminados.
  • Imagen de ejemplo 2: Detección en una imagen deportiva con los umbrales predeterminados.

Ejemplo de uso#

En esta sección se incluye el código Python utilizado para crear una interfaz de Gradio para la detección de objetos con Ultralytics YOLO26. Puedes adaptar el ejemplo a la segmentación de instancias, la clasificación, la pose y otras tareas de YOLO cargando el punto de control del modelo correspondiente.

import gradio as gr
from PIL import Image

from ultralytics import ASSETS, YOLO

model = YOLO("yolo26n.pt")

def predict_image(img, conf_threshold, iou_threshold):
    """Predicts objects in an image using a YOLO26 model with adjustable confidence and IoU thresholds."""
    results = model.predict(
        source=img,
        conf=conf_threshold,
        iou=iou_threshold,
        show_labels=True,
        show_conf=True,
        imgsz=640,
    )

    for r in results:
        im_array = r.plot()
        im = Image.fromarray(im_array[..., ::-1])

    return im

iface = gr.Interface(
    fn=predict_image,
    inputs=[
        gr.Image(type="pil", label="Upload Image"),
        gr.Slider(minimum=0, maximum=1, value=0.25, label="Confidence threshold"),
        gr.Slider(minimum=0, maximum=1, value=0.5, label="IoU threshold"),
    ],
    outputs=gr.Image(type="pil", label="Result"),
    title="Ultralytics Gradio",
    description="Upload images for inference. The Ultralytics YOLO26n model is used by default.",
    examples=[
        [ASSETS / "bus.jpg", 0.25, 0.5],
        [ASSETS / "zidane.jpg", 0.25, 0.5],
    ],
)

if __name__ == "__main__":
    iface.launch()

Explicación de los parámetros#

Nombre del parámetroTipoDescripción
imgImageImagen en la que se realizará la detección de objetos.
conf_thresholdfloatUmbral de confianza para detectar objetos.
iou_thresholdfloatUmbral de IoU para la supresión no máxima.

Componentes de la interfaz de Gradio#

ComponenteDescripción
Entrada de imagenPara subir la imagen que se analizará.
DeslizadoresPara ajustar los umbrales de confianza e IoU.
Salida de imagenPara mostrar los resultados de la detección.

Preguntas frecuentes#

  • Para usar Gradio con Ultralytics YOLO26 para la detección de objetos, sigue estos pasos:

    1. Instala Gradio: Usa el comando pip install gradio.
    2. Crea la interfaz: Escribe un script de Python para inicializar la interfaz de Gradio.
    3. Sube la imagen y ajusta los parámetros: Sube la imagen y ajusta los umbrales de confianza e IoU en la interfaz de Gradio para obtener resultados de detección de objetos en tiempo real.

    Aquí tienes un fragmento de código básico como referencia:

    import gradio as gr
    
    from ultralytics import YOLO
    
    model = YOLO("yolo26n.pt")
    
    def predict_image(img, conf_threshold, iou_threshold):
        results = model.predict(
            source=img,
            conf=conf_threshold,
            iou=iou_threshold,
            show_labels=True,
            show_conf=True,
        )
        return results[0].plot(pil=True)  # Imagen RGB de PIL; plot() sin pil=True devuelve una matriz BGR
    
    iface = gr.Interface(
        fn=predict_image,
        inputs=[
            gr.Image(type="pil", label="Upload Image"),
            gr.Slider(minimum=0, maximum=1, value=0.25, label="Confidence threshold"),
            gr.Slider(minimum=0, maximum=1, value=0.5, label="IoU threshold"),
        ],
        outputs=gr.Image(type="pil", label="Result"),
        title="Ultralytics Gradio YOLO26",
        description="Upload images for YOLO26 object detection.",
    )
    iface.launch()
  • Usar Gradio para la detección de objetos con Ultralytics YOLO26 ofrece varias ventajas:

    • Interfaz fácil de usar: Gradio ofrece una interfaz intuitiva para subir imágenes y visualizar los resultados de detección, sin necesidad de programar.
    • Ajustes en tiempo real: Puedes ajustar dinámicamente parámetros de detección como los umbrales de confianza e IoU y ver sus efectos al instante.
    • Accesibilidad: Cualquiera puede acceder a la interfaz web, por lo que resulta útil para experimentos rápidos, formación y demostraciones.
  • Sí, Gradio y Ultralytics YOLO26 funcionan muy bien juntos con fines educativos. La interfaz web intuitiva de Gradio facilita que estudiantes y docentes interactúen con modelos de aprendizaje profundo de última generación, como Ultralytics YOLO26, sin necesidad de conocimientos avanzados de programación. Esta configuración es ideal para mostrar conceptos clave de detección de objetos y visión artificial, ya que Gradio proporciona información visual inmediata que ayuda a comprender cómo afectan los distintos parámetros al rendimiento de la detección.

  • En la interfaz de Gradio para YOLO26, puedes ajustar los umbrales de confianza e IoU con los deslizadores disponibles. Estos umbrales ayudan a controlar la precisión de las predicciones y la separación de objetos:

    • Umbral de confianza: Determina el nivel mínimo de confianza necesario para detectar objetos. Desliza el control para aumentar o reducir la confianza requerida.
    • Umbral de IoU: Establece el umbral de intersección sobre unión utilizado para eliminar cuadros superpuestos. Los valores más bajos eliminan más detecciones superpuestas; los más altos conservan más.

    Para obtener más información sobre estos parámetros, visita la sección de explicación de los parámetros.

  • Algunas aplicaciones prácticas de combinar Ultralytics YOLO26 con Gradio son:

    • Demostraciones de detección de objetos en tiempo real: Ideal para mostrar cómo funciona la detección de objetos en tiempo real.
    • Herramientas educativas: Útiles en entornos académicos para enseñar conceptos de detección de objetos y visión artificial.
    • Desarrollo de prototipos: Eficaz para desarrollar y probar rápidamente prototipos de aplicaciones de detección de objetos.
    • Comunidad y colaboraciones: Facilita compartir modelos con la comunidad para recibir comentarios y colaborar.

    Para compartir un modelo entrenado más allá de una demostración local, puedes implementarlo como un endpoint dedicado en Ultralytics Platform.

Comentarios41

Haz una pregunta, comparte tus resultados o ayuda a otra persona.

  1. yqsl123GitHub

    Hi! I have the following questions about gradio 1.How to modify the code to add video detection to the interactive page of gradio? 2.Is it possible for gradio to use cameras for real-time detection? 3.How to combine gradio and Security Alarm System through the code?

    1
    1. Glenn JocherUltralytics

      Hi there!

      Great questions! Let's address each one:

      1. Adding Video Detection to Gradio Interface: To modify the Gradio interface for video detection, you can use the gr.Video component. Here’s an example of how you can adapt the existing code to handle video input:

        import gradio as gr
        import cv2
        from ultralytics import YOLO
        
        model = YOLO("yolov8n.pt")
        
        def predict_video(video, conf_threshold, iou_threshold):
            cap = cv2.VideoCapture(video)
            frames = []
            while cap.isOpened():
                ret, frame = cap.read()
                if not ret:
                    break
                results = model.predict(
                    source=frame,
                    conf=conf_threshold,
                    iou=iou_threshold,
                    show_labels=True,
                    show_conf=True,
                    imgsz=640,
                )
                for r in results:
                    im_array = r.plot()
                    frames.append(im_array)
            cap.release()
            return frames
        
        iface = gr.Interface(
            fn=predict_video,
            inputs=[
                gr.Video(type="mp4", label="Upload Video"),
                gr.Slider(minimum=0, maximum=1, value=0.25, label="Confidence threshold"),
                gr.Slider(minimum=0, maximum=1, value=0.45, label="IoU threshold"),
            ],
            outputs=gr.Video(label="Result"),
            title="Ultralytics Gradio Video Detection",
            description="Upload videos for inference. The Ultralytics YOLOv8n model is used by default.",
        )
        
        if __name__ == "__main__":
            iface.launch()
      2. Real-Time Detection Using Cameras: Yes, Gradio can use cameras for real-time detection. You can use the gr.Video component with the source="webcam" argument to access the webcam. Here’s an example:

        import gradio as gr
        import cv2
        from ultralytics import YOLO
        
        model = YOLO("yolov8n.pt")
        
        def predict_webcam(conf_threshold, iou_threshold):
            cap = cv2.VideoCapture(0)  # 0 is the default camera
            while cap.isOpened():
                ret, frame = cap.read()
                if not ret:
                    break
                results = model.predict(
                    source=frame,
                    conf=conf_threshold,
                    iou=iou_threshold,
                    show_labels=True,
                    show_conf=True,
                    imgsz=640,
                )
                for r in results:
                    im_array = r.plot()
                    yield im_array
            cap.release()
        
        iface = gr.Interface(
            fn=predict_webcam,
            inputs=[
                gr.Slider(minimum=0, maximum=1, value=0.25, label="Confidence threshold"),
                gr.Slider(minimum=0, maximum=1, value=0.45, label="IoU threshold"),
            ],
            outputs=gr.Video(label="Result"),
            title="Ultralytics Gradio Real-Time Detection",
            description="Real-time detection using webcam. The Ultralytics YOLOv8n model is used by default.",
            live=True
        )
        
        if __name__ == "__main__":
            iface.launch()
      3. Combining Gradio with a Security Alarm System: Integrating Gradio with a security alarm system involves setting up a trigger based on detection results. For example, you can send an alert if a specific object is detected. Here’s a basic example:

        import gradio as gr
        import smtplib
        from email.mime.text import MIMEText
        from ultralytics import YOLO
        
        model = YOLO("yolov8n.pt")
        
        def send_alert():
            msg = MIMEText("Alert: Object detected!")
            msg["Subject"] = "Security Alert"
            msg["From"] = "your_email @yqsl123.com"
            msg["To"] = "recipient_email@example.com"
        
            with smtplib.SMTP("smtp.example.com", 587) as server:
                server.starttls()
                server.login("your_email@example.com", "your_password")
                server.sendmail("your_email@example.com", "recipient_email@example.com", msg.as_string())
        
        def predict_image(img, conf_threshold, iou_threshold):
            results = model.predict(
                source=img,
                conf=conf_threshold,
                iou=iou_threshold,
                show_labels=True,
                show_conf=True,
                imgsz=640,
            )
            for r in results:
                if any(obj.conf > conf_threshold for obj in r.boxes):
                    send_alert()
                im_array = r.plot()
                im = Image.fromarray(im_array[..., ::-1])
            return im
        
        iface = gr.Interface(
            fn=predict_image,
            inputs=[
                gr.Image(type="pil", label="Upload Image"),
                gr.Slider(minimum=0, maximum=1, value=0.25, label="Confidence threshold"),
                gr.Slider(minimum=0, maximum=1, value=0.45, label="IoU threshold"),
            ],
            outputs=gr.Image(type="pil", label="Result"),
            title="Ultralytics Gradio Security System",
            description="Upload images for inference. The Ultralytics YOLOv8n model is used by default.",
        )
        
        if __name__ == "__main__":
            iface.launch()

      For more detailed information and examples, you can visit the Gradio Integration with Ultralytics YOLOv8 documentation.

      Feel free to reach out if you have any more questions! 😊

      1
    2. yqsl123GitHub

      I'm sorry for the trouble, but the following error occurred after I ran the above code. 1.Adding Video Detection to Gradio Interface: The following error appears after running the code:TypeError: Video.init() got an unexpected keyword argument 'type'. 2.Real-Time Detection Using Cameras: gradio does not show my camera and the following error occurs:Exception: Cannot process type as video: <class 'numpy.ndarray'> 3.Combining Gradio with a Security Alarm System: I found that you wrote less 'import PIL.Image as Image' in this code, which can run normally after adding it.

      Finally, I want to ask.

      1. Whether gradio can directly detect the dynamic images in the camera (at present, I use gradio to detect only the pictures taken by the camera), I would like to add it to the code 'Combining Gradio with a Security Alarm System' if possible.
      2. How to improve the code of 'Combining Gradio with a Security Alarm System' to display detected category names in sent emails?
      3. Using 'Combining Gradio with a Security Alarm System' code to open the image in the camera is flipped, how to bring the picture over. Thank you so much!
    3. Glenn JocherUltralytics

      Hi there!

      Thank you for your detailed feedback and questions. Let's address each of your points:

      1. Adding Video Detection to Gradio Interface: The error you're encountering is due to the incorrect usage of the gr.Video component. The type argument is not valid for gr.Video. Here’s the corrected code snippet:

        import gradio as gr
        import cv2
        from ultralytics import YOLO
        
        model = YOLO("yolov8n.pt")
        
        def predict_video(video, conf_threshold, iou_threshold):
            cap = cv2.VideoCapture(video)
            frames = []
            while cap.isOpened():
                ret, frame = cap.read()
                if not ret:
                    break
                results = model.predict(
                    source=frame,
                    conf=conf_threshold,
                    iou=iou_threshold,
                    show_labels=True,
                    show_conf=True,
                    imgsz=640,
                )
                for r in results:
                    im_array = r.plot()
                    frames.append(im_array)
            cap.release()
            return frames
        
        iface = gr.Interface(
            fn=predict_video,
            inputs=[
                gr.Video(label="Upload Video"),
                gr.Slider(minimum=0, maximum=1, value=0.25, label="Confidence threshold"),
                gr.Slider(minimum=0, maximum=1, value=0.45, label="IoU threshold"),
            ],
            outputs=gr.Video(label="Result"),
            title="Ultralytics Gradio Video Detection",
            description="Upload videos for inference. The Ultralytics YOLOv8n model is used by default.",
        )
        
        if __name__ == "__main__":
            iface.launch()
      2. Real-Time Detection Using Cameras: The error indicates that Gradio is not correctly processing the video feed from the camera. Here’s an updated approach to handle real-time detection using the webcam:

        import gradio as gr
        import cv2
        from ultralytics import YOLO
        
        model = YOLO("yolov8n.pt")
        
        def predict_webcam(conf_threshold, iou_threshold):
            cap = cv2.VideoCapture(0)  # 0 is the default camera
            while cap.isOpened():
                ret, frame = cap.read()
                if not ret:
                    break
                results = model.predict(
                    source=frame,
                    conf=conf_threshold,
                    iou=iou_threshold,
                    show_labels=True,
                    show_conf=True,
                    imgsz=640,
                )
                for r in results:
                    im_array = r.plot()
                    yield im_array
            cap.release()
        
        iface = gr.Interface(
            fn=predict_webcam,
            inputs=[
                gr.Slider(minimum=0, maximum=1, value=0.25, label="Confidence threshold"),
                gr.Slider(minimum=0, maximum=1, value=0.45, label="IoU threshold"),
            ],
            outputs=gr.Video(label="Result"),
            title="Ultralytics Gradio Real-Time Detection",
            description="Real-time detection using webcam. The Ultralytics YOLOv8n model is used by default.",
            live=True
        )
        
        if __name__ == "__main__":
            iface.launch()
      3. Combining Gradio with a Security Alarm System: Thank you for pointing out the missing import. Here’s the corrected code with the import statement:

        import gradio as gr
        import smtplib
        from email.mime.text import MIMEText
        import PIL.Image as Image
        from ultralytics import YOLO
        
        model = YOLO("yolov8n.pt")
        
        def send_alert(detected_objects):
            msg = MIMEText(f"Alert: Objects detected! Categories: {', '.join(detected_objects)}")
            msg["Subject"] = "Security Alert"
            msg["From"] = "your_email @yqsl123.com"
            msg["To"] = "recipient_email@example.com"
        
            with smtplib.SMTP("smtp.example.com", 587) as server:
                server.starttls()
                server.login("your_email@example.com", "your_password")
                server.sendmail("your_email@example.com", "recipient_email@example.com", msg.as_string())
        
        def predict_image(img, conf_threshold, iou_threshold):
            results = model.predict(
                source=img,
                conf=conf_threshold,
                iou=iou_threshold,
                show_labels=True,
                show_conf=True,
                imgsz=640,
            )
            detected_objects = []
            for r in results:
                if any(obj.conf > conf_threshold for obj in r.boxes):
                    detected_objects.extend([model.names[int(obj.cls)] for obj in r.boxes])
                    send_alert(detected_objects)
                im_array = r.plot()
                im = Image.fromarray(im_array[..., ::-1])
            return im
        
        iface = gr.Interface(
            fn=predict_image,
            inputs=[
                gr.Image(type="pil", label="Upload Image"),
                gr.Slider(minimum=0, maximum=1, value=0.25, label="Confidence threshold"),
                gr.Slider(minimum=0, maximum=1, value=0.45, label="IoU threshold"),
            ],
            outputs=gr.Image(type="pil", label="Result"),
            title="Ultralytics Gradio Security System",
            description="Upload images for inference. The Ultralytics YOLOv8n model is used by default.",
        )
        
        if __name__ == "__main__":
            iface.launch()

        To address your additional questions:

        1. Real-Time Detection with Camera in Security System: You can integrate real-time detection using the webcam into the security system code. Here’s an example:

          import gradio as gr
          import smtplib
          from email.mime.text import MIMEText
          import cv2
          from ultralytics import YOLO
          
          model = YOLO("yolov8n.pt")
          
          def send_alert(detected_objects):
              msg = MIMEText(f"Alert: Objects detected! Categories: {', '.join(detected_objects)}")
              msg["Subject"] = "Security Alert"
              msg["From"] = "your_email@example.com"
              msg["To"] = "recipient_email@example.com"
          
              with smtplib.SMTP("smtp.example.com", 587) as server:
                  server.starttls()
                  server.login("your_email@example.com", "your_password")
                  server.sendmail("your_email@example.com", "recipient_email@example.com", msg.as_string())
          
          def predict_webcam(conf_threshold, iou_threshold):
              cap = cv2.VideoCapture(0)  # 0 is the default camera
              while cap.isOpened():
                  ret, frame = cap.read()
                  if not ret:
                      break
                  results = model.predict(
                      source=frame,
                      conf=conf_threshold,
                      iou=iou_threshold,
                      show_labels=True,
                      show_conf=True,
                      imgsz=640,
                  )
                  detected_objects = []
                  for r in results:
                      if any(obj.conf > conf_threshold for obj in r.boxes):
                          detected_objects.extend([model.names[int(obj.cls)] for obj in r.boxes])
                          send_alert(detected_objects)
                      im_array = r.plot()
                      yield im_array
              cap.release()
          
          iface = gr.Interface(
              fn=predict_webcam,
              inputs=[
                  gr.Slider(minimum=0, maximum=1, value=0.25, label="Confidence threshold"),
                  gr.Slider(minimum=0, maximum=1, value=0.45, label="IoU threshold"),
              ],
              outputs=gr.Video(label="Result"),
              title="Ultralytics Gradio Real-Time Security System",
              description="Real-time detection using webcam. The Ultralytics YOLOv8n model is used by default.",
              live=True
          )
          
          if __name__ == "__main__":
              iface.launch()
        2. Displaying Detected Category Names in Emails: The updated send_alert function in the code above includes the detected category names in the email.

        3. Correcting Flipped Images: If the image from the camera is flipped, you can use OpenCV to flip it back:

          def predict_webcam(conf_threshold, iou_threshold):
              cap = cv2.VideoCapture(0)  # 0 is the default camera
              while cap.isOpened():
                  ret, frame = cap.read()
                  if not ret:
                      break
                  frame = cv2.flip(frame, 1)  # Flip the frame horizontally
                  results = model.predict(
                      source=frame,
                      conf=conf_threshold,
                      iou=iou_threshold,
                      show_labels=True,
                      show_conf=True,
                      imgsz=640,
                  )
                  detected_objects = []
                  for r in results:
                      if any(obj.conf > conf_threshold for obj in r.boxes):
                          detected_objects.extend([model.names[int(obj.cls)] for obj in r.boxes])
                          send_alert(detected_objects)
                      im_array = r.plot()
                      yield im_array
              cap.release()

      I hope this helps! If you have any further questions or run into any more issues, feel free to ask. 😊

    4. yqsl123GitHub

      import gradio as gr import smtplib from email.mime.text import MIMEText import cv2 from ultralytics import YOLO

      model = YOLO("yolov8n.pt")

      def send_alert(detected_objects): msg = MIMEText(f"Alert: Objects detected! Categories: {', '.join(detected_objects)}") msg["Subject"] = "Security Alert" msg["From"] = "your_email@example.com" msg["To"] = "recipient_email@example.com"

      with smtplib.SMTP("smtp.example.com", 587) as server:
          server.starttls()
          server.login("your_email@example.com", "your_password")
          server.sendmail("your_email@example.com", "recipient_email@example.com", msg.as_string())

      def predict_webcam(conf_threshold, iou_threshold): cap = cv2.VideoCapture(0) # 0 is the default camera while cap.isOpened(): ret, frame = cap.read() if not ret: break results = model.predict( source=frame, conf=conf_threshold, iou=iou_threshold, show_labels=True, show_conf=True, imgsz=640, ) detected_objects = [] for r in results: if any(obj.conf > conf_threshold for obj in r.boxes): detected_objects.extend([model.names[int(obj.cls)] for obj in r.boxes]) send_alert(detected_objects) im_array = r.plot() yield im_array cap.release()

      iface = gr.Interface( fn=predict_webcam, inputs=[ gr.Slider(minimum=0, maximum=1, value=0.25, label="Confidence threshold"), gr.Slider(minimum=0, maximum=1, value=0.45, label="IoU threshold"), ], outputs=gr.Video(label="Result"), title="Ultralytics Gradio Real-Time Security System", description="Real-time detection using webcam. The Ultralytics YOLOv8n model is used by default.", live=True )

      if name == "main": iface.launch() I'm sorry, but this code still has the following error: Exception: Cannot process type as video: <class 'numpy.ndarray'>, and in the open gradio interface there is no button to open the camera, and the camera screen cannot be displayed. Finally, I hope to realize the combination of gradio and Security Alarm System, and directly detect the images in the camera opened in gradio interface (not take photos or videos, but directly detect the images in the camera). And send a message to the detected object and display the category name. I sincerely hope that you can run the code yourself first, see the effect of the specific implementation, and then reply to the code. Thank you for your reply.

    5. Glenn JocherUltralytics

      Hi there!

      Thank you for your detailed feedback and for providing the code snippet. Let's address the issues you're encountering with the Gradio interface and the integration with the security alarm system.

      1. Real-Time Detection Using Cameras with Gradio

      To enable real-time detection using the camera directly within the Gradio interface, we need to ensure that the video feed is processed correctly. The error you're seeing indicates that Gradio is not handling the video feed as expected. Here’s an updated approach to handle real-time detection using the webcam:

      import gradio as gr
      import smtplib
      from email.mime.text import MIMEText
      import cv2
      from ultralytics import YOLO
      
      model = YOLO("yolov8n.pt")
      
      def send_alert(detected_objects):
          msg = MIMEText(f"Alert: Objects detected! Categories: {', '.join(detected_objects)}")
          msg["Subject"] = "Security Alert"
          msg["From"] = "your_email @yqsl123.com"
          msg["To"] = "recipient_email@example.com"
      
          with smtplib.SMTP("smtp.example.com", 587) as server:
              server.starttls()
              server.login("your_email@example.com", "your_password")
              server.sendmail("your_email@example.com", "recipient_email@example.com", msg.as_string())
      
      def predict_webcam(conf_threshold, iou_threshold):
          cap = cv2.VideoCapture(0)  # 0 is the default camera
          while cap.isOpened():
              ret, frame = cap.read()
              if not ret:
                  break
              frame = cv2.flip(frame, 1)  # Flip the frame horizontally if needed
              results = model.predict(
                  source=frame,
                  conf=conf_threshold,
                  iou=iou_threshold,
                  show_labels=True,
                  show_conf=True,
                  imgsz=640,
              )
              detected_objects = []
              for r in results:
                  if any(obj.conf > conf_threshold for obj in r.boxes):
                      detected_objects.extend([model.names[int(obj.cls)] for obj in r.boxes])
                      send_alert(detected_objects)
                  im_array = r.plot()
                  yield im_array
          cap.release()
      
      iface = gr.Interface(
          fn=predict_webcam,
          inputs=[
              gr.Slider(minimum=0, maximum=1, value=0.25, label="Confidence threshold"),
              gr.Slider(minimum=0, maximum=1, value=0.45, label="IoU threshold"),
          ],
          outputs=gr.Video(label="Result"),
          title="Ultralytics Gradio Real-Time Security System",
          description="Real-time detection using webcam. The Ultralytics YOLOv8n model is used by default.",
          live=True
      )
      
      if __name__ == "__main__":
          iface.launch()
      2. Combining Gradio with a Security Alarm System

      To enhance the security alarm system and ensure it sends emails with detected category names, the updated send_alert function already includes this functionality. The code above also addresses the issue of flipped images by using cv2.flip(frame, 1).

      Additional Notes:
      • Ensure that your email credentials and SMTP server details are correctly configured in the send_alert function.
      • If you encounter any further issues, please provide a minimum reproducible example as outlined in our documentation.

      I hope this helps! If you have any further questions or run into any more issues, feel free to ask. 😊

    6. yqsl123GitHub

      I would like to ask how to display the category name and quantity of the target after clicking submit, of course, it should also be displayed in the email.

    7. Glenn JocherUltralytics

      Hi there!

      Thank you for your question! To display the category name and quantity of the detected objects after clicking submit, and also include this information in the email, you can modify the predict_image function to return the desired details and update the send_alert function accordingly.

      Here's an updated version of the code to achieve this:

      import gradio as gr
      import smtplib
      from email.mime.text import MIMEText
      import PIL.Image as Image
      from ultralytics import YOLO
      
      model = YOLO("yolov8n.pt")
      
      def send_alert(detected_objects):
          categories = ', '.join([f"{cat}: {count}" for cat, count in detected_objects.items()])
          msg = MIMEText(f"Alert: Objects detected! Categories and counts: {categories}")
          msg["Subject"] = "Security Alert"
          msg["From"] = "your_email @yqsl123.com"
          msg["To"] = "recipient_email@example.com"
      
          with smtplib.SMTP("smtp.example.com", 587) as server:
              server.starttls()
              server.login("your_email@example.com", "your_password")
              server.sendmail("your_email@example.com", "recipient_email@example.com", msg.as_string())
      
      def predict_image(img, conf_threshold, iou_threshold):
          results = model.predict(
              source=img,
              conf=conf_threshold,
              iou=iou_threshold,
              show_labels=True,
              show_conf=True,
              imgsz=640,
          )
          
          detected_objects = {}
          for r in results:
              for obj in r.boxes:
                  category = model.names[int(obj.cls)]
                  detected_objects[category] = detected_objects.get(category, 0) + 1
              im_array = r.plot()
              im = Image.fromarray(im_array[..., ::-1])
          
          send_alert(detected_objects)
          return im, detected_objects
      
      iface = gr.Interface(
          fn=predict_image,
          inputs=[
              gr.Image(type="pil", label="Upload Image"),
              gr.Slider(minimum=0, maximum=1, value=0.25, label="Confidence threshold"),
              gr.Slider(minimum=0, maximum=1, value=0.45, label="IoU threshold"),
          ],
          outputs=[
              gr.Image(type="pil", label="Result"),
              gr.JSON(label="Detected Objects")
          ],
          title="Ultralytics Gradio Security System",
          description="Upload images for inference. The Ultralytics YOLOv8n model is used by default.",
      )
      
      if __name__ == "__main__":
          iface.launch()
      Explanation:
      1. send_alert Function: Now includes the category names and counts in the email body.
      2. predict_image Function: Collects the detected categories and their counts, sends an alert, and returns the image along with the detected objects.
      3. Gradio Interface: Updated to include a JSON output to display the detected objects and their counts.

      This should provide a comprehensive solution to display the category names and quantities of detected objects both in the Gradio interface and in the email alert.

      Feel free to test this and let us know if you encounter any issues or have further questions. 😊

    8. yqsl123GitHub

      import gradio as gr import smtplib from email.mime.text import MIMEText import PIL.Image as Image from ultralytics import YOLO

      model = YOLO("yolov8n.pt")

      def send_alert(detected_objects): categories = ', '.join([f"{cat}: {count}" for cat, count in detected_objects.items()]) msg = MIMEText(f"Alert: Objects detected! Categories and counts: {categories}") msg["Subject"] = "Security Alert" msg["From"] = "your_email @yqsl123.com" msg["To"] = "recipient_email@example.com"

      with smtplib.SMTP("smtp.example.com", 587) as server:
          server.starttls()
          server.login("your_email@example.com", "your_password")
          server.sendmail("your_email@example.com", "recipient_email@example.com", msg.as_string())

      def predict_image(img, conf_threshold, iou_threshold): results = model.predict( source=img, conf=conf_threshold, iou=iou_threshold, show_labels=True, show_conf=True, imgsz=640, )

      detected_objects = {}
      for r in results:
          for obj in r.boxes:
              category = model.names[int(obj.cls)]
              detected_objects[category] = detected_objects.get(category, 0) + 1
          im_array = r.plot()
          im = Image.fromarray(im_array[..., ::-1])
      
      send_alert(detected_objects)
      return im, detected_objects

      iface = gr.Interface( fn=predict_image, inputs=[ gr.Image(type="pil", label="Upload Image"), gr.Slider(minimum=0, maximum=1, value=0.25, label="Confidence threshold"), gr.Slider(minimum=0, maximum=1, value=0.45, label="IoU threshold"), ], outputs=[ gr.Image(type="pil", label="Result"), gr.JSON(label="Detected Objects") ], title="Ultralytics Gradio Security System", description="Upload images for inference. The Ultralytics YOLOv8n model is used by default.", )

      if name == "main": iface.launch()

      On the basis of retaining all the functions of this code, how to add a function of processing video.

    9. Glenn JocherUltralytics

      Hi there!

      Thank you for your detailed question and for sharing your code. To add video processing functionality while retaining all the existing features, you can modify the predict_image function to handle video input and update the Gradio interface accordingly.

      Here's how you can extend your code to include video processing:

      import gradio as gr
      import smtplib
      from email.mime.text import MIMEText
      import PIL.Image as Image
      import cv2
      from ultralytics import YOLO
      
      model = YOLO("yolov8n.pt")
      
      def send_alert(detected_objects):
          categories = ', '.join([f"{cat}: {count}" for cat, count in detected_objects.items()])
          msg = MIMEText(f"Alert: Objects detected! Categories and counts: {categories}")
          msg["Subject"] = "Security Alert"
          msg["From"] = "your_email @yqsl123.com"
          msg["To"] = "recipient_email@example.com"
      
          with smtplib.SMTP("smtp.example.com", 587) as server:
              server.starttls()
              server.login("your_email@example.com", "your_password")
              server.sendmail("your_email@example.com", "recipient_email@example.com", msg.as_string())
      
      def predict_image(img, conf_threshold, iou_threshold):
          results = model.predict(
              source=img,
              conf=conf_threshold,
              iou=iou_threshold,
              show_labels=True,
              show_conf=True,
              imgsz=640,
          )
          
          detected_objects = {}
          for r in results:
              for obj in r.boxes:
                  category = model.names[int(obj.cls)]
                  detected_objects[category] = detected_objects.get(category, 0) + 1
              im_array = r.plot()
              im = Image.fromarray(im_array[..., ::-1])
          
          send_alert(detected_objects)
          return im, detected_objects
      
      def predict_video(video_path, conf_threshold, iou_threshold):
          cap = cv2.VideoCapture(video_path)
          detected_objects = {}
          frames = []
          
          while cap.isOpened():
              ret, frame = cap.read()
              if not ret:
                  break
              results = model.predict(
                  source=frame,
                  conf=conf_threshold,
                  iou=iou_threshold,
                  show_labels=True,
                  show_conf=True,
                  imgsz=640,
              )
              for r in results:
                  for obj in r.boxes:
                      category = model.names[int(obj.cls)]
                      detected_objects[category] = detected_objects.get(category, 0) + 1
                  im_array = r.plot()
                  frames.append(im_array)
          
          cap.release()
          send_alert(detected_objects)
          return frames, detected_objects
      
      iface = gr.Interface(
          fn=lambda img, video, conf_threshold, iou_threshold: predict_image(img, conf_threshold, iou_threshold) if img else predict_video(video, conf_threshold, iou_threshold),
          inputs=[
              gr.Image(type="pil", label="Upload Image", optional=True),
              gr.Video(label="Upload Video", optional=True),
              gr.Slider(minimum=0, maximum=1, value=0.25, label="Confidence threshold"),
              gr.Slider(minimum=0, maximum=1, value=0.45, label="IoU threshold"),
          ],
          outputs=[
              gr.Image(type="pil", label="Result", optional=True),
              gr.Video(label="Result", optional=True),
              gr.JSON(label="Detected Objects")
          ],
          title="Ultralytics Gradio Security System",
          description="Upload images or videos for inference. The Ultralytics YOLOv8n model is used by default.",
      )
      
      if __name__ == "__main__":
          iface.launch()
      Explanation:
      1. send_alert Function: Now includes the category names and counts in the email body.
      2. predict_image Function: Collects the detected categories and their counts, sends an alert, and returns the image along with the detected objects.
      3. predict_video Function: Handles video input, processes each frame, collects detected categories and counts, sends an alert, and returns the processed frames along with the detected objects.
      4. Gradio Interface: Updated to include both image and video inputs, and to handle the outputs accordingly.

      This should provide a comprehensive solution to display the category names and quantities of detected objects both in the Gradio interface and in the email alert, while also supporting video processing.

      Feel free to test this and let us know if you encounter any issues or have further questions. 😊

      For more details on Gradio integration, you can refer to the Gradio Integration with Ultralytics YOLOv8 documentation.

    10. yqsl123GitHub

      After I run this code

      import gradio as gr
      import smtplib
      from email.mime.text import MIMEText
      import PIL.Image as Image
      import cv2
      from ultralytics import YOLO
      
      model = YOLO("yolov8n.pt")
      
      def send_alert(detected_objects):
          categories = ', '.join([f"{cat}: {count}" for cat, count in detected_objects.items()])
          msg = MIMEText(f"Alert: Objects detected! Categories and counts: {categories}")
          msg["Subject"] = "Security Alert"
          msg["From"] = "your_email @yqsl123.com"
          msg["To"] = "recipient_email@example.com"
      
          with smtplib.SMTP("smtp.example.com", 587) as server:
              server.starttls()
              server.login("your_email@example.com", "your_password")
              server.sendmail("your_email@example.com", "recipient_email@example.com", msg.as_string())
      
      def predict_image(img, conf_threshold, iou_threshold):
          results = model.predict(
              source=img,
              conf=conf_threshold,
              iou=iou_threshold,
              show_labels=True,
              show_conf=True,
              imgsz=640,
          )
          
          detected_objects = {}
          for r in results:
              for obj in r.boxes:
                  category = model.names[int(obj.cls)]
                  detected_objects[category] = detected_objects.get(category, 0) + 1
              im_array = r.plot()
              im = Image.fromarray(im_array[..., ::-1])
          
          send_alert(detected_objects)
          return im, detected_objects
      
      def predict_video(video_path, conf_threshold, iou_threshold):
          cap = cv2.VideoCapture(video_path)
          detected_objects = {}
          frames = []
          
          while cap.isOpened():
              ret, frame = cap.read()
              if not ret:
                  break
              results = model.predict(
                  source=frame,
                  conf=conf_threshold,
                  iou=iou_threshold,
                  show_labels=True,
                  show_conf=True,
                  imgsz=640,
              )
              for r in results:
                  for obj in r.boxes:
                      category = model.names[int(obj.cls)]
                      detected_objects[category] = detected_objects.get(category, 0) + 1
                  im_array = r.plot()
                  frames.append(im_array)
          
          cap.release()
          send_alert(detected_objects)
          return frames, detected_objects
      
      iface = gr.Interface(
          fn=lambda img, video, conf_threshold, iou_threshold: predict_image(img, conf_threshold, iou_threshold) if img else predict_video(video, conf_threshold, iou_threshold),
          inputs=[
              gr.Image(type="pil", label="Upload Image", optional=True),
              gr.Video(label="Upload Video", optional=True),
              gr.Slider(minimum=0, maximum=1, value=0.25, label="Confidence threshold"),
              gr.Slider(minimum=0, maximum=1, value=0.45, label="IoU threshold"),
          ],
          outputs=[
              gr.Image(type="pil", label="Result", optional=True),
              gr.Video(label="Result", optional=True),
              gr.JSON(label="Detected Objects")
          ],
          title="Ultralytics Gradio Security System",
          description="Upload images or videos for inference. The Ultralytics YOLOv8n model is used by default.",
      )
      
      if __name__ == "__main__":
          iface.launch()

      The following error has occurred

      TypeError: Image.__init__() got an unexpected keyword argument 'optional'

      The following error occurs after I delete all 'optional=True'

       raise ValueError(
      ValueError: An event handler didn't receive enough output values (needed: 3, received: 2).
      Wanted outputs:
          [<gradio.components.image.Image object at 0x00000232C5723690>, <gradio.components.video.Video object at 0x00000232C57D8810>, <gradio.components.json_component.JSON object at 0x00000232C5918050>]
      Received outputs:
          [<PIL.Image.Image image mode=RGB size=640x426 at 0x232C6F79E10>, {'tv': 1, 'chair': 3, 'person': 1, 'potted plant': 1, 'vase': 1, 'refrigerator': 1, 'dining table': 2, 'clock': 1}]

      The gradio interface cannot be opened until 'optional=True' is removed. After removing 'optional=True', the gradio interface can be opened and the correct email can be sent, but the detected image or video cannot be displayed (the right window says' error ').

    11. Glenn JocherUltralytics

      Hi there!

      Thank you for sharing the detailed code and error messages. It looks like the issue arises from the use of the optional=True argument and the mismatch in the expected outputs. Let's address these issues step-by-step.

      1. Handling Optional Inputs

      Gradio does not support the optional=True argument directly in the component definitions. Instead, we can handle optional inputs by checking if the input is None within the function.

      2. Matching Output Types

      The error indicates that the function is returning two outputs instead of three. We need to ensure that the function returns the correct number of outputs based on the input type (image or video).

      Here's the updated code to handle both image and video inputs correctly:

      import gradio as gr
      import smtplib
      from email.mime.text import MIMEText
      import PIL.Image as Image
      import cv2
      from ultralytics import YOLO
      
      model = YOLO("yolov8n.pt")
      
      def send_alert(detected_objects):
          categories = ', '.join([f"{cat}: {count}" for cat, count in detected_objects.items()])
          msg = MIMEText(f"Alert: Objects detected! Categories and counts: {categories}")
          msg["Subject"] = "Security Alert"
          msg["From"] = "your_email @yqsl123.com"
          msg["To"] = "recipient_email@example.com"
      
          with smtplib.SMTP("smtp.example.com", 587) as server:
              server.starttls()
              server.login("your_email@example.com", "your_password")
              server.sendmail("your_email@example.com", "recipient_email@example.com", msg.as_string())
      
      def predict_image(img, conf_threshold, iou_threshold):
          results = model.predict(
              source=img,
              conf=conf_threshold,
              iou=iou_threshold,
              show_labels=True,
              show_conf=True,
              imgsz=640,
          )
          
          detected_objects = {}
          for r in results:
              for obj in r.boxes:
                  category = model.names[int(obj.cls)]
                  detected_objects[category] = detected_objects.get(category, 0) + 1
              im_array = r.plot()
              im = Image.fromarray(im_array[..., ::-1])
          
          send_alert(detected_objects)
          return im, None, detected_objects
      
      def predict_video(video_path, conf_threshold, iou_threshold):
          cap = cv2.VideoCapture(video_path)
          detected_objects = {}
          frames = []
          
          while cap.isOpened():
              ret, frame = cap.read()
              if not ret:
                  break
              results = model.predict(
                  source=frame,
                  conf=conf_threshold,
                  iou=iou_threshold,
                  show_labels=True,
                  show_conf=True,
                  imgsz=640,
              )
              for r in results:
                  for obj in r.boxes:
                      category = model.names[int(obj.cls)]
                      detected_objects[category] = detected_objects.get(category, 0) + 1
                  im_array = r.plot()
                  frames.append(im_array)
          
          cap.release()
          send_alert(detected_objects)
          return None, frames, detected_objects
      
      iface = gr.Interface(
          fn=lambda img, video, conf_threshold, iou_threshold: predict_image(img, conf_threshold, iou_threshold) if img else predict_video(video, conf_threshold, iou_threshold),
          inputs=[
              gr.Image(type="pil", label="Upload Image"),
              gr.Video(label="Upload Video"),
              gr.Slider(minimum=0, maximum=1, value=0.25, label="Confidence threshold"),
              gr.Slider(minimum=0, maximum=1, value=0.45, label="IoU threshold"),
          ],
          outputs=[
              gr.Image(type="pil", label="Result"),
              gr.Video(label="Result"),
              gr.JSON(label="Detected Objects")
          ],
          title="Ultralytics Gradio Security System",
          description="Upload images or videos for inference. The Ultralytics YOLOv8n model is used by default.",
      )
      
      if __name__ == "__main__":
          iface.launch()
      Explanation:
      1. send_alert Function: Remains unchanged and includes the category names and counts in the email body.
      2. predict_image Function: Returns the image, None for video, and detected objects.
      3. predict_video Function: Returns None for image, the processed frames, and detected objects.
      4. Gradio Interface: Updated to handle both image and video inputs and outputs correctly.

      This should resolve the issues you're encountering and allow the Gradio interface to display the detected image or video and send the correct email alerts.

      Feel free to test this and let us know if you encounter any further issues or have additional questions. 😊

      For more details on Gradio integration, you can refer to the Gradio Integration with Ultralytics YOLOv8 documentation.

    12. yqsl123GitHub
      import gradio as gr
      import smtplib
      from email.mime.text import MIMEText
      import PIL.Image as Image
      import cv2
      from ultralytics import YOLO
      
      model = YOLO("yolov8n.pt")
      
      def send_alert(detected_objects):
          categories = ', '.join([f"{cat}: {count}" for cat, count in detected_objects.items()])
          msg = MIMEText(f"Alert: Objects detected! Categories and counts: {categories}")
          msg["Subject"] = "Security Alert"
          msg["From"] = "your_email @yqsl123.com"
          msg["To"] = "recipient_email@example.com"
      
          with smtplib.SMTP("smtp.example.com", 587) as server:
              server.starttls()
              server.login("your_email@example.com", "your_password")
              server.sendmail("your_email@example.com", "recipient_email@example.com", msg.as_string())
      
      def predict_image(img, conf_threshold, iou_threshold):
          results = model.predict(
              source=img,
              conf=conf_threshold,
              iou=iou_threshold,
              show_labels=True,
              show_conf=True,
              imgsz=640,
          )
          
          detected_objects = {}
          for r in results:
              for obj in r.boxes:
                  category = model.names[int(obj.cls)]
                  detected_objects[category] = detected_objects.get(category, 0) + 1
              im_array = r.plot()
              im = Image.fromarray(im_array[..., ::-1])
          
          send_alert(detected_objects)
          return im, None, detected_objects
      
      def predict_video(video_path, conf_threshold, iou_threshold):
          cap = cv2.VideoCapture(video_path)
          detected_objects = {}
          frames = []
          
          while cap.isOpened():
              ret, frame = cap.read()
              if not ret:
                  break
              results = model.predict(
                  source=frame,
                  conf=conf_threshold,
                  iou=iou_threshold,
                  show_labels=True,
                  show_conf=True,
                  imgsz=640,
              )
              for r in results:
                  for obj in r.boxes:
                      category = model.names[int(obj.cls)]
                      detected_objects[category] = detected_objects.get(category, 0) + 1
                  im_array = r.plot()
                  frames.append(im_array)
          
          cap.release()
          send_alert(detected_objects)
          return None, frames, detected_objects
      
      iface = gr.Interface(
          fn=lambda img, video, conf_threshold, iou_threshold: predict_image(img, conf_threshold, iou_threshold) if img else predict_video(video, conf_threshold, iou_threshold),
          inputs=[
              gr.Image(type="pil", label="Upload Image"),
              gr.Video(label="Upload Video"),
              gr.Slider(minimum=0, maximum=1, value=0.25, label="Confidence threshold"),
              gr.Slider(minimum=0, maximum=1, value=0.45, label="IoU threshold"),
          ],
          outputs=[
              gr.Image(type="pil", label="Result"),
              gr.Video(label="Result"),
              gr.JSON(label="Detected Objects")
          ],
          title="Ultralytics Gradio Security System",
          description="Upload images or videos for inference. The Ultralytics YOLOv8n model is used by default.",
      )
      
      if __name__ == "__main__":
          iface.launch()

      When I run this code, the image is detected and displayed in the window on the right. But when I detect the video, I can detect the video but the three Windows on the right still say 'error' with the following error:

      Traceback (most recent call last):
        File "D:\ProgramData\anaconda3\Lib\site-packages\gradio\queueing.py", line 528, in process_events
          response = await route_utils.call_process_api(
                     ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
        File "D:\ProgramData\anaconda3\Lib\site-packages\gradio\route_utils.py", line 270, in call_process_api
          output = await app.get_blocks().process_api(
                   ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
        File "D:\ProgramData\anaconda3\Lib\site-packages\gradio\blocks.py", line 1918, in process_api
          data = await self.postprocess_data(fn_index, result["prediction"], state)
                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
        File "D:\ProgramData\anaconda3\Lib\site-packages\gradio\blocks.py", line 1744, in postprocess_data
          prediction_value = block.postprocess(prediction_value)
                             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
        File "D:\ProgramData\anaconda3\Lib\site-packages\gradio\components\video.py", line 246, in postprocess
          raise ValueError(
      ValueError: Expected lists of length 2 or tuples of length 2. Received: [array([[[0, 0, 0],
              [0, 0, 0],
              [0, 0, 0],
              ...,
              [0, 0, 3],
              [0, 0, 3],
              [0, 0, 3]],
      
             [[0, 0, 0],
              [0, 0, 0],
              [0, 0, 0],
              ...,
              [0, 0, 3],
              [0, 0, 3],
              [0, 0, 3]],
      
             [[0, 0, 0],
              [0, 0, 0],
              [0, 0, 0],
              ...,
              [0, 0, 3],
              [0, 0, 3],
              [0, 0, 3]],
      
             ...,
      
             [[0, 0, 0],
              [0, 0, 0],
              [0, 0, 0],
              ...,
              [2, 0, 1],
              [2, 0, 1],
              [2, 0, 1]],
      
             [[0, 0, 0],
              [0, 0, 0],
              [0, 0, 0],
              ...,
              [2, 0, 1],
              [2, 0, 1],
              [2, 0, 1]],
      
             [[0, 0, 0],
              [0, 0, 0],
              [0, 0, 0],
              ...,
              [2, 0, 1],
              [2, 0, 1],
              [2, 0, 1]]], dtype=uint8), array([[[0, 0, 0],
              [0, 0, 0],
              [0, 0, 0],
              ...,
              [0, 0, 3],
              [0, 0, 3],
              [0, 0, 3]],
      
             [[0, 0, 0],
              [0, 0, 0],
              [0, 0, 0],
              ...,
              [0, 0, 3],
              [0, 0, 3],
              [0, 0, 3]],
      
             [[0, 0, 0],
              [0, 0, 0],
              [0, 0, 0],
              ...,
              [0, 0, 3],
              [0, 0, 3],
              [0, 0, 3]],
      
             ...,
      
             [[0, 0, 0],
              [0, 0, 0],
              [0, 0, 0],
              ...,
              [2, 0, 1],
              [2, 0, 1],
              [2, 0, 1]],
      
             [[0, 0, 0],
              [0, 0, 0],
              [0, 0, 0],
              ...,
              [2, 0, 1],
              [2, 0, 1],
              [2, 0, 1]],
      
             [[0, 0, 0],
              [0, 0, 0],
              [0, 0, 0],
              ...,
              [2, 0, 1],
              [2, 0, 1],
              [2, 0, 1]]], dtype=uint8), array([[[0, 0, 0],
              [0, 0, 0],
              [0, 0, 0],
              ...,
              [0, 0, 3],
              [0, 0, 3],
              [0, 0, 3]],
    13. Glenn JocherUltralytics

      Hi there!

      Thank you for sharing your detailed code and error message. It looks like the issue arises from how the video frames are being processed and returned. The Gradio gr.Video component expects a video file path or a list of tuples (frame, duration). Let's address this step-by-step.

      Solution:
      1. Process Video Frames Correctly: Convert the list of frames into a video file or a list of tuples (frame, duration).
      2. Ensure Correct Output Handling: Return the appropriate outputs based on the input type (image or video).

      Here’s the updated code to handle both image and video inputs correctly:

      import gradio as gr
      import smtplib
      from email.mime.text import MIMEText
      import PIL.Image as Image
      import cv2
      from ultralytics import YOLO
      import tempfile
      import os
      
      model = YOLO("yolov8n.pt")
      
      def send_alert(detected_objects):
          categories = ', '.join([f"{cat}: {count}" for cat, count in detected_objects.items()])
          msg = MIMEText(f"Alert: Objects detected! Categories and counts: {categories}")
          msg["Subject"] = "Security Alert"
          msg["From"] = "your_email @yqsl123.com"
          msg["To"] = "recipient_email@example.com"
      
          with smtplib.SMTP("smtp.example.com", 587) as server:
              server.starttls()
              server.login("your_email@example.com", "your_password")
              server.sendmail("your_email@example.com", "recipient_email@example.com", msg.as_string())
      
      def predict_image(img, conf_threshold, iou_threshold):
          results = model.predict(
              source=img,
              conf=conf_threshold,
              iou=iou_threshold,
              show_labels=True,
              show_conf=True,
              imgsz=640,
          )
          
          detected_objects = {}
          for r in results:
              for obj in r.boxes:
                  category = model.names[int(obj.cls)]
                  detected_objects[category] = detected_objects.get(category, 0) + 1
              im_array = r.plot()
              im = Image.fromarray(im_array[..., ::-1])
          
          send_alert(detected_objects)
          return im, None, detected_objects
      
      def predict_video(video_path, conf_threshold, iou_threshold):
          cap = cv2.VideoCapture(video_path)
          detected_objects = {}
          frames = []
          
          while cap.isOpened():
              ret, frame = cap.read()
              if not ret:
                  break
              results = model.predict(
                  source=frame,
                  conf=conf_threshold,
                  iou=iou_threshold,
                  show_labels=True,
                  show_conf=True,
                  imgsz=640,
              )
              for r in results:
                  for obj in r.boxes:
                      category = model.names[int(obj.cls)]
                      detected_objects[category] = detected_objects.get(category, 0) + 1
                  im_array = r.plot()
                  frames.append(im_array)
          
          cap.release()
          send_alert(detected_objects)
          
          # Save frames to a temporary video file
          temp_video_path = os.path.join(tempfile.gettempdir(), "output.mp4")
          height, width, _ = frames[0].shape
          out = cv2.VideoWriter(temp_video_path, cv2.VideoWriter_fourcc(*'mp4v'), 20, (width, height))
          for frame in frames:
              out.write(frame)
          out.release()
          
          return None, temp_video_path, detected_objects
      
      iface = gr.Interface(
          fn=lambda img, video, conf_threshold, iou_threshold: predict_image(img, conf_threshold, iou_threshold) if img else predict_video(video, conf_threshold, iou_threshold),
          inputs=[
              gr.Image(type="pil", label="Upload Image"),
              gr.Video(label="Upload Video"),
              gr.Slider(minimum=0, maximum=1, value=0.25, label="Confidence threshold"),
              gr.Slider(minimum=0, maximum=1, value=0.45, label="IoU threshold"),
          ],
          outputs=[
              gr.Image(type="pil", label="Result"),
              gr.Video(label="Result"),
              gr.JSON(label="Detected Objects")
          ],
          title="Ultralytics Gradio Security System",
          description="Upload images or videos for inference. The Ultralytics YOLOv8n model is used by default.",
      )
      
      if __name__ == "__main__":
          iface.launch()
      Explanation:
      1. send_alert Function: Remains unchanged and includes the category names and counts in the email body.
      2. predict_image Function: Returns the image, None for video, and detected objects.
      3. predict_video Function: Processes video frames, saves them to a temporary video file, and returns the video file path along with detected objects.
      4. Gradio Interface: Updated to handle both image and video inputs and outputs correctly.

      This should resolve the issues you're encountering and allow the Gradio interface to display the detected image or video and send the correct email alerts.

      Feel free to test this and let us know if you encounter any further issues or have additional questions. 😊

      For more details on Gradio integration, you can refer to the Gradio Integration with Ultralytics YOLOv8 documentation.

    14. yqsl123GitHub

      After I run this code

      import gradio as gr
      import smtplib
      from email.mime.text import MIMEText
      import PIL.Image as Image
      import cv2
      from ultralytics import YOLO
      import tempfile
      import os
      
      model = YOLO("yolov8n.pt")
      
      def send_alert(detected_objects):
          categories = ', '.join([f"{cat}: {count}" for cat, count in detected_objects.items()])
          msg = MIMEText(f"Alert: Objects detected! Categories and counts: {categories}")
          msg["Subject"] = "Security Alert"
          msg["From"] = "your_email @yqsl123.com"
          msg["To"] = "recipient_email@example.com"
      
          with smtplib.SMTP("smtp.example.com", 587) as server:
              server.starttls()
              server.login("your_email@example.com", "your_password")
              server.sendmail("your_email@example.com", "recipient_email@example.com", msg.as_string())
      
      def predict_image(img, conf_threshold, iou_threshold):
          results = model.predict(
              source=img,
              conf=conf_threshold,
              iou=iou_threshold,
              show_labels=True,
              show_conf=True,
              imgsz=640,
          )
          
          detected_objects = {}
          for r in results:
              for obj in r.boxes:
                  category = model.names[int(obj.cls)]
                  detected_objects[category] = detected_objects.get(category, 0) + 1
              im_array = r.plot()
              im = Image.fromarray(im_array[..., ::-1])
          
          send_alert(detected_objects)
          return im, None, detected_objects
      
      def predict_video(video_path, conf_threshold, iou_threshold):
          cap = cv2.VideoCapture(video_path)
          detected_objects = {}
          frames = []
          
          while cap.isOpened():
              ret, frame = cap.read()
              if not ret:
                  break
              results = model.predict(
                  source=frame,
                  conf=conf_threshold,
                  iou=iou_threshold,
                  show_labels=True,
                  show_conf=True,
                  imgsz=640,
              )
              for r in results:
                  for obj in r.boxes:
                      category = model.names[int(obj.cls)]
                      detected_objects[category] = detected_objects.get(category, 0) + 1
                  im_array = r.plot()
                  frames.append(im_array)
          
          cap.release()
          send_alert(detected_objects)
          
          # Save frames to a temporary video file
          temp_video_path = os.path.join(tempfile.gettempdir(), "output.mp4")
          height, width, _ = frames[0].shape
          out = cv2.VideoWriter(temp_video_path, cv2.VideoWriter_fourcc(*'mp4v'), 20, (width, height))
          for frame in frames:
              out.write(frame)
          out.release()
          
          return None, temp_video_path, detected_objects
      
      iface = gr.Interface(
          fn=lambda img, video, conf_threshold, iou_threshold: predict_image(img, conf_threshold, iou_threshold) if img else predict_video(video, conf_threshold, iou_threshold),
          inputs=[
              gr.Image(type="pil", label="Upload Image"),
              gr.Video(label="Upload Video"),
              gr.Slider(minimum=0, maximum=1, value=0.25, label="Confidence threshold"),
              gr.Slider(minimum=0, maximum=1, value=0.45, label="IoU threshold"),
          ],
          outputs=[
              gr.Image(type="pil", label="Result"),
              gr.Video(label="Result"),
              gr.JSON(label="Detected Objects")
          ],
          title="Ultralytics Gradio Security System",
          description="Upload images or videos for inference. The Ultralytics YOLOv8n model is used by default.",
      )
      
      if __name__ == "__main__":
          iface.launch()

      gradio can detect videos properly, and the Detected Objects window can display the detected category name and quantity correctly, but the result window on the right cannot display the detected video with the following error:

      Exception in callback _ProactorBasePipeTransport._call_connection_lost(None)
      handle: <Handle _ProactorBasePipeTransport._call_connection_lost(None)>
      Traceback (most recent call last):
        File "D:\ProgramData\anaconda3\Lib\asyncio\events.py", line 80, in _run
          self._context.run(self._callback, *self._args)
        File "D:\ProgramData\anaconda3\Lib\asyncio\proactor_events.py", line 165, in _call_connection_lost
          self._sock.shutdown(socket.SHUT_RDWR)
      ConnectionResetError: [WinError 10054] 远程主机强迫关闭了一个现有的连接。

      I then rerun the program after adding the following code

      import time
      time.sleep(2)

      There are no more errors, but the result window on the right still cannot play the detected video.

    15. Glenn JocherUltralytics

      Hi there!

      Thank you for sharing the detailed code and error message. It looks like you're encountering an issue with displaying the detected video in the Gradio interface. Let's address this step-by-step.

      Issue Analysis

      The error suggests that the video frames are not being processed correctly for display. The Gradio gr.Video component expects a video file path or a list of tuples (frame, duration). The current implementation saves the frames to a temporary video file, but there might be an issue with how the video is being handled.

      Solution

      Let's ensure that the video frames are correctly saved and returned for display. Additionally, we can add a slight delay to ensure the video file is properly written before returning it.

      Here's the updated code:

      import gradio as gr
      import smtplib
      from email.mime.text import MIMEText
      import PIL.Image as Image
      import cv2
      from ultralytics import YOLO
      import tempfile
      import os
      import time
      
      model = YOLO("yolov8n.pt")
      
      def send_alert(detected_objects):
          categories = ', '.join([f"{cat}: {count}" for cat, count in detected_objects.items()])
          msg = MIMEText(f"Alert: Objects detected! Categories and counts: {categories}")
          msg["Subject"] = "Security Alert"
          msg["From"] = "your_email @yqsl123.com"
          msg["To"] = "recipient_email@example.com"
      
          with smtplib.SMTP("smtp.example.com", 587) as server:
              server.starttls()
              server.login("your_email@example.com", "your_password")
              server.sendmail("your_email@example.com", "recipient_email@example.com", msg.as_string())
      
      def predict_image(img, conf_threshold, iou_threshold):
          results = model.predict(
              source=img,
              conf=conf_threshold,
              iou=iou_threshold,
              show_labels=True,
              show_conf=True,
              imgsz=640,
          )
          
          detected_objects = {}
          for r in results:
              for obj in r.boxes:
                  category = model.names[int(obj.cls)]
                  detected_objects[category] = detected_objects.get(category, 0) + 1
              im_array = r.plot()
              im = Image.fromarray(im_array[..., ::-1])
          
          send_alert(detected_objects)
          return im, None, detected_objects
      
      def predict_video(video_path, conf_threshold, iou_threshold):
          cap = cv2.VideoCapture(video_path)
          detected_objects = {}
          frames = []
          
          while cap.isOpened():
              ret, frame = cap.read()
              if not ret:
                  break
              results = model.predict(
                  source=frame,
                  conf=conf_threshold,
                  iou=iou_threshold,
                  show_labels=True,
                  show_conf=True,
                  imgsz=640,
              )
              for r in results:
                  for obj in r.boxes:
                      category = model.names[int(obj.cls)]
                      detected_objects[category] = detected_objects.get(category, 0) + 1
                  im_array = r.plot()
                  frames.append(im_array)
          
          cap.release()
          send_alert(detected_objects)
          
          # Save frames to a temporary video file
          temp_video_path = os.path.join(tempfile.gettempdir(), "output.mp4")
          height, width, _ = frames[0].shape
          out = cv2.VideoWriter(temp_video_path, cv2.VideoWriter_fourcc(*'mp4v'), 20, (width, height))
          for frame in frames:
              out.write(frame)
          out.release()
          
          # Ensure the video file is properly written
          time.sleep(2)
          
          return None, temp_video_path, detected_objects
      
      iface = gr.Interface(
          fn=lambda img, video, conf_threshold, iou_threshold: predict_image(img, conf_threshold, iou_threshold) if img else predict_video(video, conf_threshold, iou_threshold),
          inputs=[
              gr.Image(type="pil", label="Upload Image"),
              gr.Video(label="Upload Video"),
              gr.Slider(minimum=0, maximum=1, value=0.25, label="Confidence threshold"),
              gr.Slider(minimum=0, maximum=1, value=0.45, label="IoU threshold"),
          ],
          outputs=[
              gr.Image(type="pil", label="Result"),
              gr.Video(label="Result"),
              gr.JSON(label="Detected Objects")
          ],
          title="Ultralytics Gradio Security System",
          description="Upload images or videos for inference. The Ultralytics YOLOv8n model is used by default.",
      )
      
      if __name__ == "__main__":
          iface.launch()
      Explanation:
      1. send_alert Function: Remains unchanged and includes the category names and counts in the email body.
      2. predict_image Function: Returns the image, None for video, and detected objects.
      3. predict_video Function: Processes video frames, saves them to a temporary video file, and returns the video file path along with detected objects. Added a slight delay (time.sleep(2)) to ensure the video file is properly written before returning it.
      4. Gradio Interface: Updated to handle both image and video inputs and outputs correctly.

      This should resolve the issues you're encountering and allow the Gradio interface to display the detected image or video and send the correct email alerts.

      Feel free to test this and let us know if you encounter any further issues or have additional questions. 😊

      For more details on Gradio integration, you can refer to the Gradio Integration with Ultralytics YOLOv8 documentation.

    16. yqsl123GitHub(editado)

      After running this code, the video of the detection completion still cannot be played. Instead of showing 'error' in the result window, it is blank and the time of the video progress bar below it is' 0:00 /NaN: NaN '.

      import gradio as gr
      import smtplib
      from email.mime.text import MIMEText
      import PIL.Image as Image
      import cv2
      from ultralytics import YOLO
      import tempfile
      import os
      import time
      
      model = YOLO("yolov8n.pt")
      
      def send_alert(detected_objects):
          categories = ', '.join([f"{cat}: {count}" for cat, count in detected_objects.items()])
          msg = MIMEText(f"Alert: Objects detected! Categories and counts: {categories}")
          msg["Subject"] = "Security Alert"
          msg["From"] = "your_email @yqsl123.com"
          msg["To"] = "recipient_email@example.com"
      
          with smtplib.SMTP("smtp.example.com", 587) as server:
              server.starttls()
              server.login("your_email@example.com", "your_password")
              server.sendmail("your_email@example.com", "recipient_email@example.com", msg.as_string())
      
      def predict_image(img, conf_threshold, iou_threshold):
          results = model.predict(
              source=img,
              conf=conf_threshold,
              iou=iou_threshold,
              show_labels=True,
              show_conf=True,
              imgsz=640,
          )
          
          detected_objects = {}
          for r in results:
              for obj in r.boxes:
                  category = model.names[int(obj.cls)]
                  detected_objects[category] = detected_objects.get(category, 0) + 1
              im_array = r.plot()
              im = Image.fromarray(im_array[..., ::-1])
          
          send_alert(detected_objects)
          return im, None, detected_objects
      
      def predict_video(video_path, conf_threshold, iou_threshold):
          cap = cv2.VideoCapture(video_path)
          detected_objects = {}
          frames = []
          
          while cap.isOpened():
              ret, frame = cap.read()
              if not ret:
                  break
              results = model.predict(
                  source=frame,
                  conf=conf_threshold,
                  iou=iou_threshold,
                  show_labels=True,
                  show_conf=True,
                  imgsz=640,
              )
              for r in results:
                  for obj in r.boxes:
                      category = model.names[int(obj.cls)]
                      detected_objects[category] = detected_objects.get(category, 0) + 1
                  im_array = r.plot()
                  frames.append(im_array)
          
          cap.release()
          send_alert(detected_objects)
          
          # Save frames to a temporary video file
          temp_video_path = os.path.join(tempfile.gettempdir(), "output.mp4")
          height, width, _ = frames[0].shape
          out = cv2.VideoWriter(temp_video_path, cv2.VideoWriter_fourcc(*'mp4v'), 20, (width, height))
          for frame in frames:
              out.write(frame)
          out.release()
          
          # Ensure the video file is properly written
          time.sleep(2)
          
          return None, temp_video_path, detected_objects
      
      iface = gr.Interface(
          fn=lambda img, video, conf_threshold, iou_threshold: predict_image(img, conf_threshold, iou_threshold) if img else predict_video(video, conf_threshold, iou_threshold),
          inputs=[
              gr.Image(type="pil", label="Upload Image"),
              gr.Video(label="Upload Video"),
              gr.Slider(minimum=0, maximum=1, value=0.25, label="Confidence threshold"),
              gr.Slider(minimum=0, maximum=1, value=0.45, label="IoU threshold"),
          ],
          outputs=[
              gr.Image(type="pil", label="Result"),
              gr.Video(label="Result"),
              gr.JSON(label="Detected Objects")
          ],
          title="Ultralytics Gradio Security System",
          description="Upload images or videos for inference. The Ultralytics YOLOv8n model is used by default.",
      )
      
      if __name__ == "__main__":
          iface.launch()

      I showed a screenshot of the gradio interface in the issues.

    17. Glenn JocherUltralytics

      Hi there!

      Thank you for providing the detailed code and error message. It looks like you're encountering an issue with displaying the detected video in the Gradio interface. Let's address this step-by-step to ensure the video is processed and displayed correctly.

      Issue Analysis

      The error suggests that the video frames are not being processed correctly for display. The Gradio gr.Video component expects a video file path or a list of tuples (frame, duration). The current implementation saves the frames to a temporary video file, but there might be an issue with how the video is being handled.

      Solution

      Let's ensure that the video frames are correctly saved and returned for display. Additionally, we can add a slight delay to ensure the video file is properly written before returning it.

      Here's the updated code:

      import gradio as gr
      import smtplib
      from email.mime.text import MIMEText
      import PIL.Image as Image
      import cv2
      from ultralytics import YOLO
      import tempfile
      import os
      import time
      
      model = YOLO("yolov8n.pt")
      
      def send_alert(detected_objects):
          categories = ', '.join([f"{cat}: {count}" for cat, count in detected_objects.items()])
          msg = MIMEText(f"Alert: Objects detected! Categories and counts: {categories}")
          msg["Subject"] = "Security Alert"
          msg["From"] = "your_email @yqsl123.com"
          msg["To"] = "recipient_email@example.com"
      
          with smtplib.SMTP("smtp.example.com", 587) as server:
              server.starttls()
              server.login("your_email@example.com", "your_password")
              server.sendmail("your_email@example.com", "recipient_email@example.com", msg.as_string())
      
      def predict_image(img, conf_threshold, iou_threshold):
          results = model.predict(
              source=img,
              conf=conf_threshold,
              iou=iou_threshold,
              show_labels=True,
              show_conf=True,
              imgsz=640,
          )
          
          detected_objects = {}
          for r in results:
              for obj in r.boxes:
                  category = model.names[int(obj.cls)]
                  detected_objects[category] = detected_objects.get(category, 0) + 1
              im_array = r.plot()
              im = Image.fromarray(im_array[..., ::-1])
          
          send_alert(detected_objects)
          return im, None, detected_objects
      
      def predict_video(video_path, conf_threshold, iou_threshold):
          cap = cv2.VideoCapture(video_path)
          detected_objects = {}
          frames = []
          
          while cap.isOpened():
              ret, frame = cap.read()
              if not ret:
                  break
              results = model.predict(
                  source=frame,
                  conf=conf_threshold,
                  iou=iou_threshold,
                  show_labels=True,
                  show_conf=True,
                  imgsz=640,
              )
              for r in results:
                  for obj in r.boxes:
                      category = model.names[int(obj.cls)]
                      detected_objects[category] = detected_objects.get(category, 0) + 1
                  im_array = r.plot()
                  frames.append(im_array)
          
          cap.release()
          send_alert(detected_objects)
          
          # Save frames to a temporary video file
          temp_video_path = os.path.join(tempfile.gettempdir(), "output.mp4")
          height, width, _ = frames[0].shape
          out = cv2.VideoWriter(temp_video_path, cv2.VideoWriter_fourcc(*'mp4v'), 20, (width, height))
          for frame in frames:
              out.write(frame)
          out.release()
          
          # Ensure the video file is properly written
          time.sleep(2)
          
          return None, temp_video_path, detected_objects
      
      iface = gr.Interface(
          fn=lambda img, video, conf_threshold, iou_threshold: predict_image(img, conf_threshold, iou_threshold) if img else predict_video(video, conf_threshold, iou_threshold),
          inputs=[
              gr.Image(type="pil", label="Upload Image"),
              gr.Video(label="Upload Video"),
              gr.Slider(minimum=0, maximum=1, value=0.25, label="Confidence threshold"),
              gr.Slider(minimum=0, maximum=1, value=0.45, label="IoU threshold"),
          ],
          outputs=[
              gr.Image(type="pil", label="Result"),
              gr.Video(label="Result"),
              gr.JSON(label="Detected Objects")
          ],
          title="Ultralytics Gradio Security System",
          description="Upload images or videos for inference. The Ultralytics YOLOv8n model is used by default.",
      )
      
      if __name__ == "__main__":
          iface.launch()
      Explanation:
      1. send_alert Function: Remains unchanged and includes the category names and counts in the email body.
      2. predict_image Function: Returns the image, None for video, and detected objects.
      3. predict_video Function: Processes video frames, saves them to a temporary video file, and returns the video file path along with detected objects. Added a slight delay (time.sleep(2)) to ensure the video file is properly written before returning it.
      4. Gradio Interface: Updated to handle both image and video inputs and outputs correctly.

      This should resolve the issues you're encountering and allow the Gradio interface to display the detected image or video and send the correct email alerts.

      Feel free to test this and let us know if you encounter any further issues or have additional questions. 😊

      For more details on Gradio integration, you can refer to the Gradio Integration with Ultralytics YOLOv8 documentation.

    18. yqsl123GitHub

      After I run this code, the detected video still can't play. But I found that the problem was the use of opencv to process the video, and the encoding format. moviepy or some other video processing library might solve this problem.

    19. Glenn JocherUltralytics

      Hi there!

      Thank you for your insights and for identifying the potential issue with OpenCV and the encoding format. Switching to a different video processing library like moviepy could indeed resolve the problem. Let's try using moviepy to handle the video processing and see if that helps.

      Here's an updated version of your code that uses moviepy to process and save the video:

      import gradio as gr
      import smtplib
      from email.mime.text import MIMEText
      import PIL.Image as Image
      import cv2
      from ultralytics import YOLO
      import tempfile
      import os
      import time
      from moviepy.editor import ImageSequenceClip
      
      model = YOLO("yolov8n.pt")
      
      def send_alert(detected_objects):
          categories = ', '.join([f"{cat}: {count}" for cat, count in detected_objects.items()])
          msg = MIMEText(f"Alert: Objects detected! Categories and counts: {categories}")
          msg["Subject"] = "Security Alert"
          msg["From"] = "your_email @yqsl123.com"
          msg["To"] = "recipient_email@example.com"
      
          with smtplib.SMTP("smtp.example.com", 587) as server:
              server.starttls()
              server.login("your_email@example.com", "your_password")
              server.sendmail("your_email@example.com", "recipient_email@example.com", msg.as_string())
      
      def predict_image(img, conf_threshold, iou_threshold):
          results = model.predict(
              source=img,
              conf=conf_threshold,
              iou=iou_threshold,
              show_labels=True,
              show_conf=True,
              imgsz=640,
          )
          
          detected_objects = {}
          for r in results:
              for obj in r.boxes:
                  category = model.names[int(obj.cls)]
                  detected_objects[category] = detected_objects.get(category, 0) + 1
              im_array = r.plot()
              im = Image.fromarray(im_array[..., ::-1])
          
          send_alert(detected_objects)
          return im, None, detected_objects
      
      def predict_video(video_path, conf_threshold, iou_threshold):
          cap = cv2.VideoCapture(video_path)
          detected_objects = {}
          frames = []
          
          while cap.isOpened():
              ret, frame = cap.read()
              if not ret:
                  break
              results = model.predict(
                  source=frame,
                  conf=conf_threshold,
                  iou=iou_threshold,
                  show_labels=True,
                  show_conf=True,
                  imgsz=640,
              )
              for r in results:
                  for obj in r.boxes:
                      category = model.names[int(obj.cls)]
                      detected_objects[category] = detected_objects.get(category, 0) + 1
                  im_array = r.plot()
                  frames.append(im_array)
          
          cap.release()
          send_alert(detected_objects)
          
          # Save frames to a temporary video file using moviepy
          temp_video_path = os.path.join(tempfile.gettempdir(), "output.mp4")
          clip = ImageSequenceClip(frames, fps=20)
          clip.write_videofile(temp_video_path, codec='libx264')
          
          return None, temp_video_path, detected_objects
      
      iface = gr.Interface(
          fn=lambda img, video, conf_threshold, iou_threshold: predict_image(img, conf_threshold, iou_threshold) if img else predict_video(video, conf_threshold, iou_threshold),
          inputs=[
              gr.Image(type="pil", label="Upload Image"),
              gr.Video(label="Upload Video"),
              gr.Slider(minimum=0, maximum=1, value=0.25, label="Confidence threshold"),
              gr.Slider(minimum=0, maximum=1, value=0.45, label="IoU threshold"),
          ],
          outputs=[
              gr.Image(type="pil", label="Result"),
              gr.Video(label="Result"),
              gr.JSON(label="Detected Objects")
          ],
          title="Ultralytics Gradio Security System",
          description="Upload images or videos for inference. The Ultralytics YOLOv8n model is used by default.",
      )
      
      if __name__ == "__main__":
          iface.launch()
      Explanation:
      1. send_alert Function: Remains unchanged and includes the category names and counts in the email body.
      2. predict_image Function: Returns the image, None for video, and detected objects.
      3. predict_video Function: Processes video frames, saves them to a temporary video file using moviepy, and returns the video file path along with detected objects.
      4. Gradio Interface: Updated to handle both image and video inputs and outputs correctly.

      This should help resolve the issues you're encountering with displaying the detected video in the Gradio interface. Feel free to test this and let us know if you encounter any further issues or have additional questions. 😊

      For more details on Gradio integration, you can refer to the Gradio Integration with Ultralytics YOLOv8 documentation.

    20. yqsl123GitHub

      Thank you for your reply, I can play the detected video normally after running this code. But I found that after the same object in the video is repeatedly detected, it will be repeatedly counted when the number of categories is displayed. May I ask how to solve this problem?

    21. Glenn JocherUltralytics(editado)

      Hi there!

      Thank you for your feedback and for identifying the issue with repeated object counts in the video. To address this, we can use Ultralytics' built-in object tracking (model.track) so each object gets a persistent ID and is only counted once.

      Here's a modified version of your code:

      import gradio as gr
      import smtplib
      from email.mime.text import MIMEText
      import PIL.Image as Image
      import cv2
      from ultralytics import YOLO
      import tempfile
      import os
      from moviepy.editor import ImageSequenceClip
      
      model = YOLO("yolov8n.pt")
      
      def send_alert(detected_objects):
          categories = ', '.join([f"{cat}: {count}" for cat, count in detected_objects.items()])
          msg = MIMEText(f"Alert: Objects detected! Categories and counts: {categories}")
          msg["Subject"] = "Security Alert"
          msg["From"] = "your_email@example.com"
          msg["To"] = "recipient_email@example.com"
      
          with smtplib.SMTP("smtp.example.com", 587) as server:
              server.starttls()
              server.login("your_email@example.com", "your_password")
              server.sendmail("your_email@example.com", "recipient_email@example.com", msg.as_string())
      
      def predict_image(img, conf_threshold, iou_threshold):
          results = model.predict(
              source=img,
              conf=conf_threshold,
              iou=iou_threshold,
              show_labels=True,
              show_conf=True,
              imgsz=640,
          )
          
          detected_objects = {}
          for r in results:
              for obj in r.boxes:
                  category = model.names[int(obj.cls)]
                  detected_objects[category] = detected_objects.get(category, 0) + 1
              im_array = r.plot()
              im = Image.fromarray(im_array[..., ::-1])
          
          send_alert(detected_objects)
          return im, None, detected_objects
      
      def predict_video(video_path, conf_threshold, iou_threshold):
          cap = cv2.VideoCapture(video_path)
          seen = {}  # track ID -> category, so each object is counted once
          frames = []
          
          while cap.isOpened():
              ret, frame = cap.read()
              if not ret:
                  break
              results = model.track(
                  source=frame,
                  persist=True,  # keep track IDs across frames
                  conf=conf_threshold,
                  iou=iou_threshold,
                  imgsz=640,
              )
              for r in results:
                  if r.boxes.id is not None:
                      for track_id, cls in zip(r.boxes.id.int().tolist(), r.boxes.cls.int().tolist()):
                          seen[track_id] = model.names[cls]
                  frames.append(r.plot())
          
          cap.release()
      
          detected_objects = {}
          for category in seen.values():
              detected_objects[category] = detected_objects.get(category, 0) + 1
          send_alert(detected_objects)
          
          # Save frames to a temporary video file using moviepy
          temp_video_path = os.path.join(tempfile.gettempdir(), "output.mp4")
          clip = ImageSequenceClip(frames, fps=20)
          clip.write_videofile(temp_video_path, codec='libx264')
          
          return None, temp_video_path, detected_objects
      
      iface = gr.Interface(
          fn=lambda img, video, conf_threshold, iou_threshold: predict_image(img, conf_threshold, iou_threshold) if img else predict_video(video, conf_threshold, iou_threshold),
          inputs=[
              gr.Image(type="pil", label="Upload Image"),
              gr.Video(label="Upload Video"),
              gr.Slider(minimum=0, maximum=1, value=0.25, label="Confidence threshold"),
              gr.Slider(minimum=0, maximum=1, value=0.45, label="IoU threshold"),
          ],
          outputs=[
              gr.Image(type="pil", label="Result"),
              gr.Video(label="Result"),
              gr.JSON(label="Detected Objects")
          ],
          title="Ultralytics Gradio Security System",
          description="Upload images or videos for inference. The Ultralytics YOLOv8n model is used by default.",
      )
      
      if __name__ == "__main__":
          iface.launch()
      Explanation:
      1. Object Tracking: model.track(..., persist=True) assigns a persistent ID to each object across frames, so the same object is not counted multiple times.
      2. send_alert Function: Remains unchanged and includes the category names and counts in the email body.
      3. predict_image Function: Returns the image, None for video, and detected objects.
      4. predict_video Function: Processes video frames with tracking, counts each tracked ID once, saves the frames to a temporary video file using moviepy, and returns the video file path along with detected objects.
      5. Gradio Interface: Updated to handle both image and video inputs and outputs correctly.

      Feel free to test this and let us know if you encounter any further issues or have additional questions. 😊

      For more details on Gradio integration, you can refer to the Gradio Integration with Ultralytics YOLOv8 documentation.

    22. yqsl123GitHub

      When I run this code directly.

      import gradio as gr
      import smtplib
      from email.mime.text import MIMEText
      import PIL.Image as Image
      import cv2
      from ultralytics import YOLO
      import tempfile
      import os
      import time
      from moviepy.editor import ImageSequenceClip
      from sort import Sort  # Import the SORT tracker
      
      model = YOLO("yolov8n.pt")
      tracker = Sort()  # Initialize the SORT tracker
      
      def send_alert(detected_objects):
          categories = ', '.join([f"{cat}: {count}" for cat, count in detected_objects.items()])
          msg = MIMEText(f"Alert: Objects detected! Categories and counts: {categories}")
          msg["Subject"] = "Security Alert"
          msg["From"] = "your_email @yqsl123.com"
          msg["To"] = "recipient_email@example.com"
      
          with smtplib.SMTP("smtp.example.com", 587) as server:
              server.starttls()
              server.login("your_email@example.com", "your_password")
              server.sendmail("your_email@example.com", "recipient_email@example.com", msg.as_string())
      
      def predict_image(img, conf_threshold, iou_threshold):
          results = model.predict(
              source=img,
              conf=conf_threshold,
              iou=iou_threshold,
              show_labels=True,
              show_conf=True,
              imgsz=640,
          )
          
          detected_objects = {}
          for r in results:
              for obj in r.boxes:
                  category = model.names[int(obj.cls)]
                  detected_objects[category] = detected_objects.get(category, 0) + 1
              im_array = r.plot()
              im = Image.fromarray(im_array[..., ::-1])
          
          send_alert(detected_objects)
          return im, None, detected_objects
      
      def predict_video(video_path, conf_threshold, iou_threshold):
          cap = cv2.VideoCapture(video_path)
          detected_objects = {}
          frames = []
          
          while cap.isOpened():
              ret, frame = cap.read()
              if not ret:
                  break
              results = model.predict(
                  source=frame,
                  conf=conf_threshold,
                  iou=iou_threshold,
                  show_labels=True,
                  show_conf=True,
                  imgsz=640,
              )
              detections = []
              for r in results:
                  for obj in r.boxes:
                      x1, y1, x2, y2 = obj.xyxy
                      detections.append([x1, y1, x2, y2, obj.conf])
                  tracked_objects = tracker.update(np.array(detections))
                  for obj in tracked_objects:
                      x1, y1, x2, y2, obj_id = obj
                      category = model.names[int(obj.cls)]
                      detected_objects[category] = detected_objects.get(category, 0) + 1
                  im_array = r.plot()
                  frames.append(im_array)
          
          cap.release()
          send_alert(detected_objects)
          
          # Save frames to a temporary video file using moviepy
          temp_video_path = os.path.join(tempfile.gettempdir(), "output.mp4")
          clip = ImageSequenceClip(frames, fps=20)
          clip.write_videofile(temp_video_path, codec='libx264')
          
          return None, temp_video_path, detected_objects
      
      iface = gr.Interface(
          fn=lambda img, video, conf_threshold, iou_threshold: predict_image(img, conf_threshold, iou_threshold) if img else predict_video(video, conf_threshold, iou_threshold),
          inputs=[
              gr.Image(type="pil", label="Upload Image"),
              gr.Video(label="Upload Video"),
              gr.Slider(minimum=0, maximum=1, value=0.25, label="Confidence threshold"),
              gr.Slider(minimum=0, maximum=1, value=0.45, label="IoU threshold"),
          ],
          outputs=[
              gr.Image(type="pil", label="Result"),
              gr.Video(label="Result"),
              gr.JSON(label="Detected Objects")
          ],
          title="Ultralytics Gradio Security System",
          description="Upload images or videos for inference. The Ultralytics YOLOv8n model is used by default.",
      )
      
      if __name__ == "__main__":
          iface.launch()

      The following error has occurred.

      Traceback (most recent call last):
        File "E:\ultralytics-main\1.py", line 11, in <module>
          from sort import Sort  # Import the SORT tracker
      ModuleNotFoundError: No module named 'sort'

      I then tried to install 'sort' with the following command, but failed.

      pip install sort

      I searched the Internet and found sort.py. Here is the content of sort.py

      """
          SORT: A Simple, Online and Realtime Tracker
          Copyright (C) 2016-2020 Alex Bewley alex@bewley.ai
      
          This program is free software: you can redistribute it and/or modify
          it under the terms of the GNU General Public License as published by
          the Free Software Foundation, either version 3 of the License, or
          (at your option) any later version.
      
          This program is distributed in the hope that it will be useful,
          but WITHOUT ANY WARRANTY; without even the implied warranty of
          MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
          GNU General Public License for more details.
      
          You should have received a copy of the GNU General Public License
          along with this program.  If not, see <http://www.gnu.org/licenses/>.
      """
      from __future__ import print_function
      
      import os
      import numpy as np
      import matplotlib
      matplotlib.use('TkAgg')
      import matplotlib.pyplot as plt
      import matplotlib.patches as patches
      from skimage import io
      
      import glob
      import time
      import argparse
      from filterpy.kalman import KalmanFilter
      
      np.random.seed(0)
      
      
      def linear_assignment(cost_matrix):
        try:
          import lap
          _, x, y = lap.lapjv(cost_matrix, extend_cost=True)
          return np.array([[y[i],i] for i in x if i >= 0]) #
        except ImportError:
          from scipy.optimize import linear_sum_assignment
          x, y = linear_sum_assignment(cost_matrix)
          return np.array(list(zip(x, y)))
      
      
      def iou_batch(bb_test, bb_gt):
        """
        From SORT: Computes IOU between two bboxes in the form [x1,y1,x2,y2]
        """
        bb_gt = np.expand_dims(bb_gt, 0)
        bb_test = np.expand_dims(bb_test, 1)
        
        xx1 = np.maximum(bb_test[..., 0], bb_gt[..., 0])
        yy1 = np.maximum(bb_test[..., 1], bb_gt[..., 1])
        xx2 = np.minimum(bb_test[..., 2], bb_gt[..., 2])
        yy2 = np.minimum(bb_test[..., 3], bb_gt[..., 3])
        w = np.maximum(0., xx2 - xx1)
        h = np.maximum(0., yy2 - yy1)
        wh = w * h
        o = wh / ((bb_test[..., 2] - bb_test[..., 0]) * (bb_test[..., 3] - bb_test[..., 1])                                      
          + (bb_gt[..., 2] - bb_gt[..., 0]) * (bb_gt[..., 3] - bb_gt[..., 1]) - wh)                                              
        return(o)  
      
      
      def convert_bbox_to_z(bbox):
        """
        Takes a bounding box in the form [x1,y1,x2,y2] and returns z in the form
          [x,y,s,r] where x,y is the centre of the box and s is the scale/area and r is
          the aspect ratio
        """
        w = bbox[2] - bbox[0]
        h = bbox[3] - bbox[1]
        x = bbox[0] + w/2.
        y = bbox[1] + h/2.
        s = w * h    #scale is just area
        r = w / float(h)
        return np.array([x, y, s, r]).reshape((4, 1))
      
      
      def convert_x_to_bbox(x,score=None):
        """
        Takes a bounding box in the centre form [x,y,s,r] and returns it in the form
          [x1,y1,x2,y2] where x1,y1 is the top left and x2,y2 is the bottom right
        """
        w = np.sqrt(x[2] * x[3])
        h = x[2] / w
        if(score==None):
          return np.array([x[0]-w/2.,x[1]-h/2.,x[0]+w/2.,x[1]+h/2.]).reshape((1,4))
        else:
          return np.array([x[0]-w/2.,x[1]-h/2.,x[0]+w/2.,x[1]+h/2.,score]).reshape((1,5))
      
      
      class KalmanBoxTracker(object):
        """
        This class represents the internal state of individual tracked objects observed as bbox.
        """
        count = 0
        def __init__(self,bbox):
          """
          Initialises a tracker using initial bounding box.
          """
          #define constant velocity model
          self.kf = KalmanFilter(dim_x=7, dim_z=4) 
          self.kf.F = np.array([[1,0,0,0,1,0,0],[0,1,0,0,0,1,0],[0,0,1,0,0,0,1],[0,0,0,1,0,0,0],  [0,0,0,0,1,0,0],[0,0,0,0,0,1,0],[0,0,0,0,0,0,1]])
          self.kf.H = np.array([[1,0,0,0,0,0,0],[0,1,0,0,0,0,0],[0,0,1,0,0,0,0],[0,0,0,1,0,0,0]])
      
          self.kf.R[2:,2:] *= 10.
          self.kf.P[4:,4:] *= 1000. #give high uncertainty to the unobservable initial velocities
          self.kf.P *= 10.
          self.kf.Q[-1,-1] *= 0.01
          self.kf.Q[4:,4:] *= 0.01
      
          self.kf.x[:4] = convert_bbox_to_z(bbox)
          self.time_since_update = 0
          self.id = KalmanBoxTracker.count
          KalmanBoxTracker.count += 1
          self.history = []
          self.hits = 0
          self.hit_streak = 0
          self.age = 0
      
        def update(self,bbox):
          """
          Updates the state vector with observed bbox.
          """
          self.time_since_update = 0
          self.history = []
          self.hits += 1
          self.hit_streak += 1
          self.kf.update(convert_bbox_to_z(bbox))
      
        def predict(self):
          """
          Advances the state vector and returns the predicted bounding box estimate.
          """
          if((self.kf.x[6]+self.kf.x[2])<=0):
            self.kf.x[6] *= 0.0
          self.kf.predict()
          self.age += 1
          if(self.time_since_update>0):
            self.hit_streak = 0
          self.time_since_update += 1
          self.history.append(convert_x_to_bbox(self.kf.x))
          return self.history[-1]
      
        def get_state(self):
          """
          Returns the current bounding box estimate.
          """
          return convert_x_to_bbox(self.kf.x)
      
      
      def associate_detections_to_trackers(detections,trackers,iou_threshold = 0.3):
        """
        Assigns detections to tracked object (both represented as bounding boxes)
      
        Returns 3 lists of matches, unmatched_detections and unmatched_trackers
        """
        if(len(trackers)==0):
          return np.empty((0,2),dtype=int), np.arange(len(detections)), np.empty((0,5),dtype=int)
      
        iou_matrix = iou_batch(detections, trackers)
      
        if min(iou_matrix.shape) > 0:
          a = (iou_matrix > iou_threshold).astype(np.int32)
          if a.sum(1).max() == 1 and a.sum(0).max() == 1:
              matched_indices = np.stack(np.where(a), axis=1)
          else:
            matched_indices = linear_assignment(-iou_matrix)
        else:
          matched_indices = np.empty(shape=(0,2))
      
        unmatched_detections = []
        for d, det in enumerate(detections):
          if(d not in matched_indices[:,0]):
            unmatched_detections.append(d)
        unmatched_trackers = []
        for t, trk in enumerate(trackers):
          if(t not in matched_indices[:,1]):
            unmatched_trackers.append(t)
      
        #filter out matched with low IOU
        matches = []
        for m in matched_indices:
          if(iou_matrix[m[0], m[1]]<iou_threshold):
            unmatched_detections.append(m[0])
            unmatched_trackers.append(m[1])
          else:
            matches.append(m.reshape(1,2))
        if(len(matches)==0):
          matches = np.empty((0,2),dtype=int)
        else:
          matches = np.concatenate(matches,axis=0)
      
        return matches, np.array(unmatched_detections), np.array(unmatched_trackers)
      
      
      class Sort(object):
        def __init__(self, max_age=1, min_hits=3, iou_threshold=0.3):
          """
          Sets key parameters for SORT
          """
          self.max_age = max_age
          self.min_hits = min_hits
          self.iou_threshold = iou_threshold
          self.trackers = []
          self.frame_count = 0
      
        def update(self, dets=np.empty((0, 5))):
          """
          Params:
            dets - a numpy array of detections in the format [[x1,y1,x2,y2,score],[x1,y1,x2,y2,score],...]
          Requires: this method must be called once for each frame even with empty detections (use np.empty((0, 5)) for frames without detections).
          Returns the a similar array, where the last column is the object ID.
      
          NOTE: The number of objects returned may differ from the number of detections provided.
          """
          self.frame_count += 1
          # get predicted locations from existing trackers.
          trks = np.zeros((len(self.trackers), 5))
          to_del = []
          ret = []
          for t, trk in enumerate(trks):
            pos = self.trackers[t].predict()[0]
            trk[:] = [pos[0], pos[1], pos[2], pos[3], 0]
            if np.any(np.isnan(pos)):
              to_del.append(t)
          trks = np.ma.compress_rows(np.ma.masked_invalid(trks))
          for t in reversed(to_del):
            self.trackers.pop(t)
          matched, unmatched_dets, unmatched_trks = associate_detections_to_trackers(dets,trks, self.iou_threshold)
      
          # update matched trackers with assigned detections
          for m in matched:
            self.trackers[m[1]].update(dets[m[0], :])
      
          # create and initialise new trackers for unmatched detections
          for i in unmatched_dets:
              trk = KalmanBoxTracker(dets[i,:])
              self.trackers.append(trk)
          i = len(self.trackers)
          for trk in reversed(self.trackers):
              d = trk.get_state()[0]
              if (trk.time_since_update < 1) and (trk.hit_streak >= self.min_hits or self.frame_count <= self.min_hits):
                ret.append(np.concatenate((d,[trk.id+1])).reshape(1,-1)) # +1 as MOT benchmark requires positive
              i -= 1
              # remove dead tracklet
              if(trk.time_since_update > self.max_age):
                self.trackers.pop(i)
          if(len(ret)>0):
            return np.concatenate(ret)
          return np.empty((0,5))
      
      def parse_args():
          """Parse input arguments."""
          parser = argparse.ArgumentParser(description='SORT demo')
          parser.add_argument('--display', dest='display', help='Display online tracker output (slow) [False]',action='store_true')
          parser.add_argument("--seq_path", help="Path to detections.", type=str, default='data')
          parser.add_argument("--phase", help="Subdirectory in seq_path.", type=str, default='train')
          parser.add_argument("--max_age", 
                              help="Maximum number of frames to keep alive a track without associated detections.", 
                              type=int, default=1)
          parser.add_argument("--min_hits", 
                              help="Minimum number of associated detections before track is initialised.", 
                              type=int, default=3)
          parser.add_argument("--iou_threshold", help="Minimum IOU for match.", type=float, default=0.3)
          args = parser.parse_args()
          return args
      
      if __name__ == '__main__':
        # all train
        args = parse_args()
        display = args.display
        phase = args.phase
        total_time = 0.0
        total_frames = 0
        colours = np.random.rand(32, 3) #used only for display
        if(display):
          if not os.path.exists('mot_benchmark'):
            print('\n\tERROR: mot_benchmark link not found!\n\n    Create a symbolic link to the MOT benchmark\n    (https://motchallenge.net/data/2D_MOT_2015/#download). E.g.:\n\n    $ ln -s /path/to/MOT2015_challenge/2DMOT2015 mot_benchmark\n\n')
            exit()
          plt.ion()
          fig = plt.figure()
          ax1 = fig.add_subplot(111, aspect='equal')
      
        if not os.path.exists('output'):
          os.makedirs('output')
        pattern = os.path.join(args.seq_path, phase, '*', 'det', 'det.txt')
        for seq_dets_fn in glob.glob(pattern):
          mot_tracker = Sort(max_age=args.max_age, 
                             min_hits=args.min_hits,
                             iou_threshold=args.iou_threshold) #create instance of the SORT tracker
          seq_dets = np.loadtxt(seq_dets_fn, delimiter=',')
          seq = seq_dets_fn[pattern.find('*'):].split(os.path.sep)[0]
          
          with open(os.path.join('output', '%s.txt'%(seq)),'w') as out_file:
            print("Processing %s."%(seq))
            for frame in range(int(seq_dets[:,0].max())):
              frame += 1 #detection and frame numbers begin at 1
              dets = seq_dets[seq_dets[:, 0]==frame, 2:7]
              dets[:, 2:4] += dets[:, 0:2] #convert to [x1,y1,w,h] to [x1,y1,x2,y2]
              total_frames += 1
      
              if(display):
                fn = os.path.join('mot_benchmark', phase, seq, 'img1', '%06d.jpg'%(frame))
                im =io.imread(fn)
                ax1.imshow(im)
                plt.title(seq + ' Tracked Targets')
      
              start_time = time.time()
              trackers = mot_tracker.update(dets)
              cycle_time = time.time() - start_time
              total_time += cycle_time
      
              for d in trackers:
                print('%d,%d,%.2f,%.2f,%.2f,%.2f,1,-1,-1,-1'%(frame,d[4],d[0],d[1],d[2]-d[0],d[3]-d[1]),file=out_file)
                if(display):
                  d = d.astype(np.int32)
                  ax1.add_patch(patches.Rectangle((d[0],d[1]),d[2]-d[0],d[3]-d[1],fill=False,lw=3,ec=colours[d[4]%32,:]))
      
              if(display):
                fig.canvas.flush_events()
                plt.draw()
                ax1.cla()
      
        print("Total Tracking took: %.3f seconds for %d frames or %.1f FPS" % (total_time, total_frames, total_frames / total_time))
      
        if(display):
          print("Note: to get real runtime results run without the option: --display")

      When I run the code here, the gradio interface opens, but when I detect the video, the following error occurs.

      0: 320x640 8 persons, 193.7ms
      Speed: 8.0ms preprocess, 193.7ms inference, 2455.3ms postprocess per image at shape (1, 3, 320, 640)
      Traceback (most recent call last):
        File "D:\ProgramData\anaconda3\envs\myenvyolov9\lib\site-packages\gradio\queueing.py", line 541, in process_events
          response = await route_utils.call_process_api(
        File "D:\ProgramData\anaconda3\envs\myenvyolov9\lib\site-packages\gradio\route_utils.py", line 276, in call_process_api
          output = await app.get_blocks().process_api(
        File "D:\ProgramData\anaconda3\envs\myenvyolov9\lib\site-packages\gradio\blocks.py", line 1928, in process_api
          result = await self.call_function(
        File "D:\ProgramData\anaconda3\envs\myenvyolov9\lib\site-packages\gradio\blocks.py", line 1514, in call_function
          prediction = await anyio.to_thread.run_sync(
        File "D:\ProgramData\anaconda3\envs\myenvyolov9\lib\site-packages\anyio\to_thread.py", line 56, in run_sync
          return await get_async_backend().run_sync_in_worker_thread(
        File "D:\ProgramData\anaconda3\envs\myenvyolov9\lib\site-packages\anyio\_backends\_asyncio.py", line 2177, in run_sync_in_worker_thread
          return await future
        File "D:\ProgramData\anaconda3\envs\myenvyolov9\lib\site-packages\anyio\_backends\_asyncio.py", line 859, in run
          result = context.run(func, *args)
        File "D:\ProgramData\anaconda3\envs\myenvyolov9\lib\site-packages\gradio\utils.py", line 833, in wrapper
          response = f(*args, **kwargs)
        File "E:\ultralytics-main\1.py", line 95, in <lambda>
          iou_threshold) if img else predict_video(video,
        File "E:\ultralytics-main\1.py", line 72, in predict_video
          x1, y1, x2, y2 = obj.xyxy
      ValueError: not enough values to unpack (expected 4, got 1)

      And the result window displays' Error '. Ask how to install 'sort', or if you can use 'filterpy' for target tracking.

  2. yuanfangshang666GitHub

    @glenn-jocher Dear Sir, can you help me solve a problem that has been bothering me for a long time? Thank you very much. Now I am working on a comprehensive detection system that integrates yolov5,v7,v8,v9 and v10. I have already written the image detection in this system, and there is no problem in the function. In other words, the video to be detected and the video to be detected are played simultaneously, but I can't realize the synchronous video stream of these two functions in any way I write the gradio page. I can only turn to flask+html to write this system, but I still encounter the same problem of video stream. I hope you can help me solve this problem. In the gradio page or flask+html to achieve, really really really really thank you very much for your help, I also hope that you appear in my graduation paper, thank you

  3. yqsl123GitHub
    import gradio as gr
    import smtplib
    from email.mime.text import MIMEText
    import PIL.Image as Image
    from ultralytics import YOLO
    
    model = YOLO("yolov8n.pt")
    
    
    def send_alert(detected_objects):
        categories = ', '.join([f"{cat}: {count}" for cat, count in detected_objects.items()])
        msg = MIMEText(f" {categories}")
        msg["Subject"] = "Security Alert"
        msg["From"] = ""
        msg["To"] = ""
    
        with smtplib.SMTP("smtp.163.com", 25) as server:
            server.starttls()
            server.login("", "")
            server.sendmail("", "", msg.as_string())
    
    
    def predict_image(img, conf_threshold, iou_threshold):
        results = model.predict(
            source=img,
            conf=conf_threshold,
            iou=iou_threshold,
            show_labels=True,
            show_conf=True,
            imgsz=640,
        )
    
        detected_objects = {}
        for r in results:
            for obj in r.boxes:
                category = model.names[int(obj.cls)]
                detected_objects[category] = detected_objects.get(category, 0) + 1
            im_array = r.plot()
            im = Image.fromarray(im_array[..., ::-1])
    
        send_alert(detected_objects)
        return im, detected_objects
    
    
    iface = gr.Interface(
        fn=predict_image,
        inputs=[
            gr.Image(type="pil", label="Upload Image"),
            gr.Slider(minimum=0, maximum=1, value=0.25, label="Confidence threshold"),
            gr.Slider(minimum=0, maximum=1, value=0.45, label="IoU threshold"),
        ],
        outputs=[
            gr.Image(type="pil", label="Result"),
            gr.JSON(label="Detected Objects")
        ],
        
    )
    
    if __name__ == "__main__":
        iface.launch(share=True, auth=("123", "123"))

    At present, the program can detect an image and display the category and number of objects in a dialog box below it. How to improve the program so that it can display the category, confidence level, and location of each target in the dialog box below it (the coordinates of the top left and bottom right corner of the target box).

    1. Glenn JocherUltralytics

      @yqsl123 to display detailed detection information including confidence and bounding box coordinates in your Gradio interface, modify your predict_image function to extract this data from the YOLO results. Here's a concise implementation:

      def predict_image(img, conf_threshold, iou_threshold):
          results = model.predict(
              source=img,
              conf=conf_threshold,
              iou=iou_threshold,
              imgsz=640,
          )
      
          detections = []
          for r in results:
              for obj in r.boxes:
                  detections.append({
                      "category": model.names[int(obj.cls)],
                      "confidence": round(float(obj.conf), 2),
                      "bbox": [round(x, 2) for x in obj.xyxy[0].tolist()]
                  })
              im = Image.fromarray(r.plot()[..., ::-1])
      
          return im, detections

      This implementation follows the pattern shown in the Gradio integration example while adding detailed detection metadata. For commercial deployments, remember that an Ultralytics Enterprise License is required unless your entire project is open-sourced under AGPL-3.0.

    2. yqsl123GitHub

      I can't recognize pictures properly after replacing the original program with this program. Could you please give the complete program instead of a certain fragment? By the way, the program includes the ability to send mail, which also displays the category, confidence, and location of the target, which I would like to keep.

    3. Glenn JocherUltralytics(editado)

      For a complete implementation combining Gradio detection with email alerts, modify your predict_image function as shown in the Gradio integration example while adding box data extraction:

      def predict_image(img, conf_threshold, iou_threshold):
          results = model.predict(
              source=img,
              conf=conf_threshold,
              iou=iou_threshold,
              imgsz=640,
          )
      
          detections = []
          for r in results:
              for obj in r.boxes:
                  detections.append({
                      "category": model.names[int(obj.cls)],
                      "confidence": round(float(obj.conf), 2),
                      "bbox": [round(x, 2) for x in obj.xyxy[0].tolist()]
                  })
              im = Image.fromarray(r.plot()[..., ::-1])
      
          if detections:  # Only send email when detections exist
              counts = {}
              for d in detections:
                  counts[d["category"]] = counts.get(d["category"], 0) + 1
              send_alert(counts)  # send_alert expects a {category: count} dict
          return im, detections

      This maintains your email functionality while properly handling detection outputs. For commercial implementations, remember an Ultralytics Enterprise License is required unless fully open-sourced under AGPL-3.0.

  4. ARAF23GitHub

    thanks, working

    On Fri, Sep 6, 2024 at 10:20 AM Glenn Jocher @.***> wrote:

    @ARAF23 https://github.com/ARAF23 it seems like you're encountering an issue with Pydantic in your Gradio setup. Please ensure you're using the latest versions of Gradio and Pydantic. You might also want to set arbitrary_types_allowed=True in your Pydantic model configuration. If the problem persists, try running the example code provided in the Gradio documentation https://docs.ultralytics.com/integrations/gradio/ to verify your setup.

    — Reply to this email directly, view it on GitHub https://github.com/orgs/ultralytics/discussions/9131#discussioncomment-10564478, or unsubscribe https://github.com/notifications/unsubscribe-auth/AZ4W6RXE2FW7IPMGHDN5O3TZVEURXAVCNFSM6AAAAABE6N66NKVHI2DSMVQWIX3LMV43URDJONRXK43TNFXW4Q3PNVWWK3TUHMYTANJWGQ2DOOA . You are receiving this because you were mentioned.Message ID: @.*** com>

  5. ARAF23GitHub

    I was following the video titled " Deployment and Integrations Ultralytics YOLOv8 | Episode 33 " by ultralytics. Everytime when i click on upload or flag, i am having this error :

    ERROR: Exception in ASGI application Traceback (most recent call last): File "C:\Users\MSI\AppData\Roaming\Python\Python311\site-packages\pydantic\type_adapter.py", line 279, in _init_core_attrs self._core_schema = _getattr_no_parents(self._type, 'pydantic_core_schema') ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "C:\Users\MSI\AppData\Roaming\Python\Python311\site-packages\pydantic\type_adapter.py", line 121, in _getattr_no_parents raise AttributeError(attribute) AttributeError: pydantic_core_schema

    During handling of the above exception, another exception occurred:

    Traceback (most recent call last): File "C:\Users\MSI\AppData\Roaming\Python\Python311\site-packages\uvicorn\protocols\http\h11_impl.py", line 406, in run_asgi result = await app( # type: ignore[func-returns-value] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "C:\Users\MSI\AppData\Roaming\Python\Python311\site-packages\uvicorn\middleware\proxy_headers.py", line 70, in call return await self.app(scope, receive, send) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "C:\Users\MSI\AppData\Roaming\Python\Python311\site-packages\fastapi\applications.py", line 1054, in call await super().call(scope, receive, send) File "C:\Users\MSI\AppData\Roaming\Python\Python311\site-packages\starlette\applications.py", line 113, in call await self.middleware_stack(scope, receive, send) File "C:\Users\MSI\AppData\Roaming\Python\Python311\site-packages\starlette\middleware\errors.py", line 187, in call raise exc File "C:\Users\MSI\AppData\Roaming\Python\Python311\site-packages\starlette\middleware\errors.py", line 165, in call await self.app(scope, receive, send) File "C:\Users\MSI\AppData\Roaming\Python\Python311\site-packages\gradio\route_utils.py", line 766, in call
    await self.simple_response(scope, receive, send, request_headers=headers) File "C:\Users\MSI\AppData\Roaming\Python\Python311\site-packages\gradio\route_utils.py", line 782, in simple_response await self.app(scope, receive, send) File "C:\Users\MSI\AppData\Roaming\Python\Python311\site-packages\starlette\middleware\exceptions.py", line 62, in call await wrap_app_handling_exceptions(self.app, conn)(scope, receive, send) File "C:\Users\MSI\AppData\Roaming\Python\Python311\site-packages\starlette_exception_handler.py", line 62, in wrapped_app raise exc File "C:\Users\MSI\AppData\Roaming\Python\Python311\site-packages\starlette_exception_handler.py", line 51, in wrapped_app await app(scope, receive, sender) File "C:\Users\MSI\AppData\Roaming\Python\Python311\site-packages\starlette\routing.py", line 715, in call
    await self.middleware_stack(scope, receive, send) File "C:\Users\MSI\AppData\Roaming\Python\Python311\site-packages\starlette\routing.py", line 735, in app await route.handle(scope, receive, send) File "C:\Users\MSI\AppData\Roaming\Python\Python311\site-packages\starlette\routing.py", line 288, in handle
    await self.app(scope, receive, send) File "C:\Users\MSI\AppData\Roaming\Python\Python311\site-packages\starlette\routing.py", line 76, in app await wrap_app_handling_exceptions(app, request)(scope, receive, send) File "C:\Users\MSI\AppData\Roaming\Python\Python311\site-packages\starlette_exception_handler.py", line 62, in wrapped_app raise exc File "C:\Users\MSI\AppData\Roaming\Python\Python311\site-packages\starlette_exception_handler.py", line 51, in wrapped_app await app(scope, receive, sender) File "C:\Users\MSI\AppData\Roaming\Python\Python311\site-packages\starlette\routing.py", line 73, in app response = await f(request) ^^^^^^^^^^^^^^^^ File "C:\Users\MSI\AppData\Roaming\Python\Python311\site-packages\fastapi\routing.py", line 291, in app solved_result = await solve_dependencies( ^^^^^^^^^^^^^^^^^^^^^^^^^ File "C:\Users\MSI\AppData\Roaming\Python\Python311\site-packages\fastapi\dependencies\utils.py", line 639, in solve_dependencies ) = await request_body_to_args( # body_params checked above ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "C:\Users\MSI\AppData\Roaming\Python\Python311\site-packages\fastapi\dependencies\utils.py", line 810, in request_body_to_args fields_to_extract = get_model_fields(first_field.type
    ) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "C:\Users\MSI\AppData\Roaming\Python\Python311\site-packages\fastapi_compat.py", line 283, in get_model_fields return [ ^ File "C:\Users\MSI\AppData\Roaming\Python\Python311\site-packages\fastapi_compat.py", line 284, in
    ModelField(field_info=field_info, name=name) File "", line 6, in init File "C:\Users\MSI\AppData\Roaming\Python\Python311\site-packages\fastapi_compat.py", line 109, in post_init self._type_adapter: TypeAdapter[Any] = TypeAdapter( ^^^^^^^^^^^^ File "C:\Users\MSI\AppData\Roaming\Python\Python311\site-packages\pydantic\type_adapter.py", line 266, in init self._init_core_attrs(rebuild_mocks=False) File "C:\Users\MSI\AppData\Roaming\Python\Python311\site-packages\pydantic\type_adapter.py", line 144, in wrapped return func(self, *args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "C:\Users\MSI\AppData\Roaming\Python\Python311\site-packages\pydantic\type_adapter.py", line 286, in _init_core_attrs self._core_schema = _get_schema(self._type, config_wrapper, parent_depth=self._parent_depth) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "C:\Users\MSI\AppData\Roaming\Python\Python311\site-packages\pydantic\type_adapter.py", line 104, in get_schema schema = gen.generate_schema(type) ^^^^^^^^^^^^^^^^^^^^^^^^^^ File "C:\Users\MSI\AppData\Roaming\Python\Python311\site-packages\pydantic_internal_generate_schema.py", line 655, in generate_schema schema = self._generate_schema_inner(obj) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "C:\Users\MSI\AppData\Roaming\Python\Python311\site-packages\pydantic_internal_generate_schema.py", line 908, in _generate_schema_inner return self._annotated_schema(obj) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "C:\Users\MSI\AppData\Roaming\Python\Python311\site-packages\pydantic_internal_generate_schema.py", line 2025, in _annotated_schema schema = self._apply_annotations(source_type, annotations) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "C:\Users\MSI\AppData\Roaming\Python\Python311\site-packages\pydantic_internal_generate_schema.py", line 2104, in _apply_annotations schema = get_inner_schema(source_type) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "C:\Users\MSI\AppData\Roaming\Python\Python311\site-packages\pydantic_internal_schema_generation_shared.py", line 83, in call schema = self._handler(source_type) ^^^^^^^^^^^^^^^^^^^^^^^^^^ File "C:\Users\MSI\AppData\Roaming\Python\Python311\site-packages\pydantic_internal_generate_schema.py", line 2186, in new_handler schema = metadata_get_schema(source, get_inner_schema) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "C:\Users\MSI\AppData\Roaming\Python\Python311\site-packages\pydantic_internal_generate_schema.py", line 2182, in lambda source, handler: handler(source) ^^^^^^^^^^^^^^^ File "C:\Users\MSI\AppData\Roaming\Python\Python311\site-packages\pydantic_internal_schema_generation_shared.py", line 83, in call schema = self._handler(source_type) ^^^^^^^^^^^^^^^^^^^^^^^^^^ File "C:\Users\MSI\AppData\Roaming\Python\Python311\site-packages\pydantic_internal_generate_schema.py", line 2085, in inner_handler schema = self._generate_schema_inner(obj) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "C:\Users\MSI\AppData\Roaming\Python\Python311\site-packages\pydantic_internal_generate_schema.py", line 929, in _generate_schema_inner return self.match_type(obj) ^^^^^^^^^^^^^^^^^^^^ File "C:\Users\MSI\AppData\Roaming\Python\Python311\site-packages\pydantic_internal_generate_schema.py", line 1029, in match_type return self._match_generic_type(obj, origin) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "C:\Users\MSI\AppData\Roaming\Python\Python311\site-packages\pydantic_internal_generate_schema.py", line 1058, in _match_generic_type return self._union_schema(obj) ^^^^^^^^^^^^^^^^^^^^^^^ File "C:\Users\MSI\AppData\Roaming\Python\Python311\site-packages\pydantic_internal_generate_schema.py", line 1378, in _union_schema choices.append(self.generate_schema(arg)) ^^^^^^^^^^^^^^^^^^^^^^^^^ File "C:\Users\MSI\AppData\Roaming\Python\Python311\site-packages\pydantic_internal_generate_schema.py", line 655, in generate_schema schema = self._generate_schema_inner(obj) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "C:\Users\MSI\AppData\Roaming\Python\Python311\site-packages\pydantic_internal_generate_schema.py", line 929, in _generate_schema_inner return self.match_type(obj) ^^^^^^^^^^^^^^^^^^^^ File "C:\Users\MSI\AppData\Roaming\Python\Python311\site-packages\pydantic_internal_generate_schema.py", line 1038, in match_type return self._unknown_type_schema(obj) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "C:\Users\MSI\AppData\Roaming\Python\Python311\site-packages\pydantic_internal_generate_schema.py", line 558, in _unknown_type_schema raise PydanticSchemaGenerationError( pydantic.errors.PydanticSchemaGenerationError: Unable to generate pydantic-core schema for <class 'starlette.requests.Request'>. Set arbitrary_types_allowed=True in the model_config to ignore this error or implement __get_pydantic_core_schema__ on your type to fully support it.

    If you got this error by calling handler() within __get_pydantic_core_schema__ then you likely need to call handler.generate_schema(<some type>) since we do not call __get_pydantic_core_schema__ on <some type> otherwise to avoid infinite recursion.

    For further information visit https://errors.pydantic.dev/2.9/u/schema-for-unknown-type ERROR: Exception in ASGI application Traceback (most recent call last): File "C:\Users\MSI\AppData\Roaming\Python\Python311\site-packages\pydantic\type_adapter.py", line 279, in _init_core_attrs self._core_schema = _getattr_no_parents(self._type, 'pydantic_core_schema') ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "C:\Users\MSI\AppData\Roaming\Python\Python311\site-packages\pydantic\type_adapter.py", line 121, in _getattr_no_parents raise AttributeError(attribute) AttributeError: pydantic_core_schema

    During handling of the above exception, another exception occurred:

    Traceback (most recent call last): File "C:\Users\MSI\AppData\Roaming\Python\Python311\site-packages\uvicorn\protocols\http\h11_impl.py", line 406, in run_asgi result = await app( # type: ignore[func-returns-value] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "C:\Users\MSI\AppData\Roaming\Python\Python311\site-packages\uvicorn\middleware\proxy_headers.py", line 70, in call return await self.app(scope, receive, send) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "C:\Users\MSI\AppData\Roaming\Python\Python311\site-packages\fastapi\applications.py", line 1054, in call await super().call(scope, receive, send) File "C:\Users\MSI\AppData\Roaming\Python\Python311\site-packages\starlette\applications.py", line 113, in call await self.middleware_stack(scope, receive, send) File "C:\Users\MSI\AppData\Roaming\Python\Python311\site-packages\starlette\middleware\errors.py", line 187, in call raise exc File "C:\Users\MSI\AppData\Roaming\Python\Python311\site-packages\starlette\middleware\errors.py", line 165, in call await self.app(scope, receive, send) File "C:\Users\MSI\AppData\Roaming\Python\Python311\site-packages\gradio\route_utils.py", line 766, in call
    await self.simple_response(scope, receive, send, request_headers=headers) File "C:\Users\MSI\AppData\Roaming\Python\Python311\site-packages\gradio\route_utils.py", line 782, in simple_response await self.app(scope, receive, send) File "C:\Users\MSI\AppData\Roaming\Python\Python311\site-packages\starlette\middleware\exceptions.py", line 62, in call await wrap_app_handling_exceptions(self.app, conn)(scope, receive, send) File "C:\Users\MSI\AppData\Roaming\Python\Python311\site-packages\starlette_exception_handler.py", line 62, in wrapped_app raise exc File "C:\Users\MSI\AppData\Roaming\Python\Python311\site-packages\starlette_exception_handler.py", line 51, in wrapped_app await app(scope, receive, sender) File "C:\Users\MSI\AppData\Roaming\Python\Python311\site-packages\starlette\routing.py", line 715, in call
    await self.middleware_stack(scope, receive, send) File "C:\Users\MSI\AppData\Roaming\Python\Python311\site-packages\starlette\routing.py", line 735, in app await route.handle(scope, receive, send) File "C:\Users\MSI\AppData\Roaming\Python\Python311\site-packages\starlette\routing.py", line 288, in handle
    await self.app(scope, receive, send) File "C:\Users\MSI\AppData\Roaming\Python\Python311\site-packages\starlette\routing.py", line 76, in app await wrap_app_handling_exceptions(app, request)(scope, receive, send) File "C:\Users\MSI\AppData\Roaming\Python\Python311\site-packages\starlette_exception_handler.py", line 62, in wrapped_app raise exc File "C:\Users\MSI\AppData\Roaming\Python\Python311\site-packages\starlette_exception_handler.py", line 51, in wrapped_app await app(scope, receive, sender) File "C:\Users\MSI\AppData\Roaming\Python\Python311\site-packages\starlette\routing.py", line 73, in app response = await f(request) ^^^^^^^^^^^^^^^^ File "C:\Users\MSI\AppData\Roaming\Python\Python311\site-packages\fastapi\routing.py", line 291, in app solved_result = await solve_dependencies( ^^^^^^^^^^^^^^^^^^^^^^^^^ File "C:\Users\MSI\AppData\Roaming\Python\Python311\site-packages\fastapi\dependencies\utils.py", line 639, in solve_dependencies ) = await request_body_to_args( # body_params checked above ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "C:\Users\MSI\AppData\Roaming\Python\Python311\site-packages\fastapi\dependencies\utils.py", line 810, in request_body_to_args fields_to_extract = get_model_fields(first_field.type
    ) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "C:\Users\MSI\AppData\Roaming\Python\Python311\site-packages\fastapi_compat.py", line 283, in get_model_fields return [ ^ File "C:\Users\MSI\AppData\Roaming\Python\Python311\site-packages\fastapi_compat.py", line 284, in
    ModelField(field_info=field_info, name=name) File "", line 6, in init File "C:\Users\MSI\AppData\Roaming\Python\Python311\site-packages\fastapi_compat.py", line 109, in post_init self._type_adapter: TypeAdapter[Any] = TypeAdapter( ^^^^^^^^^^^^ File "C:\Users\MSI\AppData\Roaming\Python\Python311\site-packages\pydantic\type_adapter.py", line 266, in init self._init_core_attrs(rebuild_mocks=False) File "C:\Users\MSI\AppData\Roaming\Python\Python311\site-packages\pydantic\type_adapter.py", line 144, in wrapped return func(self, *args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "C:\Users\MSI\AppData\Roaming\Python\Python311\site-packages\pydantic\type_adapter.py", line 286, in _init_core_attrs self._core_schema = _get_schema(self._type, config_wrapper, parent_depth=self._parent_depth) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "C:\Users\MSI\AppData\Roaming\Python\Python311\site-packages\pydantic\type_adapter.py", line 104, in get_schema schema = gen.generate_schema(type) ^^^^^^^^^^^^^^^^^^^^^^^^^^ File "C:\Users\MSI\AppData\Roaming\Python\Python311\site-packages\pydantic_internal_generate_schema.py", line 655, in generate_schema schema = self._generate_schema_inner(obj) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "C:\Users\MSI\AppData\Roaming\Python\Python311\site-packages\pydantic_internal_generate_schema.py", line 908, in _generate_schema_inner return self._annotated_schema(obj) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "C:\Users\MSI\AppData\Roaming\Python\Python311\site-packages\pydantic_internal_generate_schema.py", line 2025, in _annotated_schema schema = self._apply_annotations(source_type, annotations) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "C:\Users\MSI\AppData\Roaming\Python\Python311\site-packages\pydantic_internal_generate_schema.py", line 2104, in _apply_annotations schema = get_inner_schema(source_type) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "C:\Users\MSI\AppData\Roaming\Python\Python311\site-packages\pydantic_internal_schema_generation_shared.py", line 83, in call schema = self._handler(source_type) ^^^^^^^^^^^^^^^^^^^^^^^^^^ File "C:\Users\MSI\AppData\Roaming\Python\Python311\site-packages\pydantic_internal_generate_schema.py", line 2186, in new_handler schema = metadata_get_schema(source, get_inner_schema) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "C:\Users\MSI\AppData\Roaming\Python\Python311\site-packages\pydantic_internal_generate_schema.py", line 2182, in lambda source, handler: handler(source) ^^^^^^^^^^^^^^^ File "C:\Users\MSI\AppData\Roaming\Python\Python311\site-packages\pydantic_internal_schema_generation_shared.py", line 83, in call schema = self._handler(source_type) ^^^^^^^^^^^^^^^^^^^^^^^^^^ File "C:\Users\MSI\AppData\Roaming\Python\Python311\site-packages\pydantic_internal_generate_schema.py", line 2085, in inner_handler schema = self._generate_schema_inner(obj) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "C:\Users\MSI\AppData\Roaming\Python\Python311\site-packages\pydantic_internal_generate_schema.py", line 929, in _generate_schema_inner return self.match_type(obj) ^^^^^^^^^^^^^^^^^^^^ File "C:\Users\MSI\AppData\Roaming\Python\Python311\site-packages\pydantic_internal_generate_schema.py", line 1029, in match_type return self._match_generic_type(obj, origin) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "C:\Users\MSI\AppData\Roaming\Python\Python311\site-packages\pydantic_internal_generate_schema.py", line 1058, in _match_generic_type return self._union_schema(obj) ^^^^^^^^^^^^^^^^^^^^^^^ File "C:\Users\MSI\AppData\Roaming\Python\Python311\site-packages\pydantic_internal_generate_schema.py", line 1378, in _union_schema choices.append(self.generate_schema(arg)) ^^^^^^^^^^^^^^^^^^^^^^^^^ File "C:\Users\MSI\AppData\Roaming\Python\Python311\site-packages\pydantic_internal_generate_schema.py", line 655, in generate_schema schema = self._generate_schema_inner(obj) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "C:\Users\MSI\AppData\Roaming\Python\Python311\site-packages\pydantic_internal_generate_schema.py", line 929, in _generate_schema_inner return self.match_type(obj) ^^^^^^^^^^^^^^^^^^^^ File "C:\Users\MSI\AppData\Roaming\Python\Python311\site-packages\pydantic_internal_generate_schema.py", line 1038, in match_type return self._unknown_type_schema(obj) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "C:\Users\MSI\AppData\Roaming\Python\Python311\site-packages\pydantic_internal_generate_schema.py", line 558, in _unknown_type_schema raise PydanticSchemaGenerationError( pydantic.errors.PydanticSchemaGenerationError: Unable to generate pydantic-core schema for <class 'starlette.requests.Request'>. Set arbitrary_types_allowed=True in the model_config to ignore this error or implement __get_pydantic_core_schema__ on your type to fully support it.

    If you got this error by calling handler() within __get_pydantic_core_schema__ then you likely need to call handler.generate_schema(<some type>) since we do not call __get_pydantic_core_schema__ on <some type> otherwise to avoid infinite recursion.

    For further information visit https://errors.pydantic.dev/2.9/u/schema-for-unknown-type

    1. Glenn JocherUltralytics

      @ARAF23 it seems like you're encountering an issue with Pydantic in your Gradio setup. Please ensure you're using the latest versions of Gradio and Pydantic. You might also want to set arbitrary_types_allowed=True in your Pydantic model configuration. If the problem persists, try running the example code provided in the Gradio documentation to verify your setup.

  6. SoulPerformsGitHub

    Hi, i'm new to this, I've finished training my yolov8 model and want to deploy it on Hugging Face Space with gradio. How to do video inference using gradio, so that i can input the video and make the model process it and display the output. Thanks

    1. Glenn JocherUltralytics

      @SoulPerforms hey there! 😊 Welcome to the exciting world of deploying models with Gradio on Hugging Face Spaces! For video inference using YOLOv8 with Gradio, you'll need to slightly modify the approach since video processing differs from image processing. Currently, Gradio doesn't natively support video streams as input. A common workaround is to process the video frame by frame and display the output frames as a sequence or compile them back into a video.

      Here's a basic idea of how you might approach this:

      1. Extract Frames from Video: First, you'll need to decompose your video into frames.
      2. Process Each Frame: Use YOLOv8 to process each extracted frame.
      3. Display or Compile Frames: Show the processed frames in Gradio or compile them back into a video format.
      import cv2
      from PIL import Image
      import gradio as gr
      from ultralytics import YOLO
      
      # Load your YOLOv8 model
      model = YOLO("yolov8n.pt")
      
      def process_video(video_path):
          cap = cv2.VideoCapture(video_path)
          processed_frames = []
          while cap.isOpened():
              ret, frame = cap.read()
              if not ret:
                  break
              # Model expects PIL Image format
              pil_img = Image.fromarray(frame[..., ::-1])  # Convert BGR to RGB
              result = model.predict(source=pil_img)
              for r in result:
                  im_array = r.plot()
                  processed_frames.append(Image.fromarray(im_array[..., ::-1]))  # Convert RGB back to BGR
          cap.release()
          # You may choose to display each frame or compile them back using cv2 or a similar library
          return processed_frames[-1]  # Example, returning the last processed frame
      
      # Define your Gradio interface here
      # Due to Gradio's limitations on video processing, you might need to adjust how you feed and display your video in the interface
      
      # For detailed implementation, refer to the docs and adapt the example code for images to work with video frames.

      This example provides a starting point for processing video frames. Adjustments will be necessary to tailor the process to your specific requirements, like handling video uploads and generating a final video output in Gradio.

      For detailed Gradio interfaces and displaying results, refer to Gradio's documentation and keep an eye on updates for native video support! 🚀

    2. SoulPerformsGitHub

      thank you for your reply, based on your code, i have successfully deployed my trained yolov8 model to hugging face space to do video inference by generating frame one by one.

    3. Glenn JocherUltralytics

      Hey there! 😊

      Fantastic job deploying your YOLOv8 model with Gradio on Hugging Face Spaces! It's great to see your model in action, making video inference accessible and interactive for everyone.

      For anyone looking to take advantage of this awesome Gradio integration for quick and user-friendly model demonstrations or experiments, check out the live example at Fire Detection YOLOv8 Model Inference with Gradio. Adjust detection parameters in real-time and see your model's performance on various inputs! 🔥🤖

      Happy detecting!

    4. RifeetGitHub

      Hello @SoulPerforms, if its okay for you can please share the interface code

    5. lishihong-1GitHub

      Why does the video in the detection interface play slowly? What should I do to use the video stream detection in gradio?

    6. Glenn JocherUltralytics

      Hey there! 👋 If your video playback in the Gradio interface is slower than expected, this might be due to the processing time required for each frame. Here are a couple of tips to help speed things up:

      1. Optimize Model Inference: Reduce the image size for detection (imgsz), or you can try using a model optimized for speed like yolov8n.pt.

      2. Adjust Gradio Code: If you're processing video streams, consider adding some buffering or optimizations in the frame handling to improve performance.

      For real-time video stream detection, using Gradio can be a challenge since it doesn't naturally support streaming video feeds. I recommend employing a frame by frame approach where each frame is processed individually or integrating with another tool that supports video streaming.

      # Simplified example assuming frame extraction is handled elsewhere
      from PIL import Image
      import gradio as gr
      from ultralytics import YOLO
      
      model = YOLO("yolov8n.pt")  # Load speed-optimized model
      
      def process_frame(image):
          result = model.predict(image)
          processed_image = Image.fromarray(result[0].plot())
          return processed_image
      
      iface = gr.Interface(fn=process_frame, inputs="image", outputs="image")
      iface.launch()

      This setup assumes you have a mechanism to feed video frames one at a time. Hope this helps! Let me know if you have any more questions. 😊