Detección interactiva de objetos: Gradio y Ultralytics YOLO26
Introducción a la detección interactiva de objetos#
Esta interfaz de Gradio ofrece una forma sencilla e interactiva de realizar detección de objetos con el modelo Ultralytics YOLO26. Puedes subir imágenes y ajustar parámetros como el umbral de confianza y el umbral de intersección sobre unión (IoU) para obtener resultados de detección en tiempo real.
Ver: Implementa Ultralytics YOLO26 en un navegador con Gradio | Tutorial completo | IA para visión 🌐
¿Por qué usar Gradio para la detección de objetos?#
- Interfaz fácil de usar: Gradio ofrece una plataforma sencilla para subir imágenes y visualizar los resultados de detección, sin necesidad de programar.
- Ajustes en tiempo real: Puedes ajustar sobre la marcha parámetros como los umbrales de confianza e IoU y obtener comentarios inmediatos para optimizar los resultados de detección.
- Amplia accesibilidad: Cualquier persona puede acceder a la interfaz web de Gradio, lo que la hace útil para demostraciones, actividades educativas y experimentos rápidos.
Cómo instalar Gradio#
pip install gradioCómo usar la interfaz#
- Sube una imagen: Haz clic en «Subir imagen» para elegir un archivo de imagen para la detección de objetos.
- Ajusta los parámetros:
- Umbral de confianza: Deslizador para establecer el nivel mínimo de confianza para detectar objetos.
- Umbral de IoU: Control deslizante para ajustar el umbral de IoU que utiliza la supresión no máxima para eliminar cuadros superpuestos.
- Consulta los resultados: Se mostrará la imagen procesada con los objetos detectados y sus etiquetas.
Ejemplos de uso#
- Imagen de ejemplo 1: Detección de autobuses con los umbrales predeterminados.
- Imagen de ejemplo 2: Detección en una imagen deportiva con los umbrales predeterminados.
Ejemplo de uso#
En esta sección se incluye el código Python utilizado para crear una interfaz de Gradio para la detección de objetos con Ultralytics YOLO26. Puedes adaptar el ejemplo a la segmentación de instancias, la clasificación, la pose y otras tareas de YOLO cargando el punto de control del modelo correspondiente.
import gradio as gr
from PIL import Image
from ultralytics import ASSETS, YOLO
model = YOLO("yolo26n.pt")
def predict_image(img, conf_threshold, iou_threshold):
"""Predicts objects in an image using a YOLO26 model with adjustable confidence and IoU thresholds."""
results = model.predict(
source=img,
conf=conf_threshold,
iou=iou_threshold,
show_labels=True,
show_conf=True,
imgsz=640,
)
for r in results:
im_array = r.plot()
im = Image.fromarray(im_array[..., ::-1])
return im
iface = gr.Interface(
fn=predict_image,
inputs=[
gr.Image(type="pil", label="Upload Image"),
gr.Slider(minimum=0, maximum=1, value=0.25, label="Confidence threshold"),
gr.Slider(minimum=0, maximum=1, value=0.5, label="IoU threshold"),
],
outputs=gr.Image(type="pil", label="Result"),
title="Ultralytics Gradio",
description="Upload images for inference. The Ultralytics YOLO26n model is used by default.",
examples=[
[ASSETS / "bus.jpg", 0.25, 0.5],
[ASSETS / "zidane.jpg", 0.25, 0.5],
],
)
if __name__ == "__main__":
iface.launch()Explicación de los parámetros#
| Nombre del parámetro | Tipo | Descripción |
|---|---|---|
img | Image | Imagen en la que se realizará la detección de objetos. |
conf_threshold | float | Umbral de confianza para detectar objetos. |
iou_threshold | float | Umbral de IoU para la supresión no máxima. |
Componentes de la interfaz de Gradio#
| Componente | Descripción |
|---|---|
| Entrada de imagen | Para subir la imagen que se analizará. |
| Deslizadores | Para ajustar los umbrales de confianza e IoU. |
| Salida de imagen | Para mostrar los resultados de la detección. |
Preguntas frecuentes#
Para usar Gradio con Ultralytics YOLO26 para la detección de objetos, sigue estos pasos:
- Instala Gradio: Usa el comando
pip install gradio. - Crea la interfaz: Escribe un script de Python para inicializar la interfaz de Gradio.
- Sube la imagen y ajusta los parámetros: Sube la imagen y ajusta los umbrales de confianza e IoU en la interfaz de Gradio para obtener resultados de detección de objetos en tiempo real.
Aquí tienes un fragmento de código básico como referencia:
import gradio as gr from ultralytics import YOLO model = YOLO("yolo26n.pt") def predict_image(img, conf_threshold, iou_threshold): results = model.predict( source=img, conf=conf_threshold, iou=iou_threshold, show_labels=True, show_conf=True, ) return results[0].plot(pil=True) # Imagen RGB de PIL; plot() sin pil=True devuelve una matriz BGR iface = gr.Interface( fn=predict_image, inputs=[ gr.Image(type="pil", label="Upload Image"), gr.Slider(minimum=0, maximum=1, value=0.25, label="Confidence threshold"), gr.Slider(minimum=0, maximum=1, value=0.5, label="IoU threshold"), ], outputs=gr.Image(type="pil", label="Result"), title="Ultralytics Gradio YOLO26", description="Upload images for YOLO26 object detection.", ) iface.launch()- Instala Gradio: Usa el comando
Usar Gradio para la detección de objetos con Ultralytics YOLO26 ofrece varias ventajas:
- Interfaz fácil de usar: Gradio ofrece una interfaz intuitiva para subir imágenes y visualizar los resultados de detección, sin necesidad de programar.
- Ajustes en tiempo real: Puedes ajustar dinámicamente parámetros de detección como los umbrales de confianza e IoU y ver sus efectos al instante.
- Accesibilidad: Cualquiera puede acceder a la interfaz web, por lo que resulta útil para experimentos rápidos, formación y demostraciones.
Sí, Gradio y Ultralytics YOLO26 funcionan muy bien juntos con fines educativos. La interfaz web intuitiva de Gradio facilita que estudiantes y docentes interactúen con modelos de aprendizaje profundo de última generación, como Ultralytics YOLO26, sin necesidad de conocimientos avanzados de programación. Esta configuración es ideal para mostrar conceptos clave de detección de objetos y visión artificial, ya que Gradio proporciona información visual inmediata que ayuda a comprender cómo afectan los distintos parámetros al rendimiento de la detección.
En la interfaz de Gradio para YOLO26, puedes ajustar los umbrales de confianza e IoU con los deslizadores disponibles. Estos umbrales ayudan a controlar la precisión de las predicciones y la separación de objetos:
- Umbral de confianza: Determina el nivel mínimo de confianza necesario para detectar objetos. Desliza el control para aumentar o reducir la confianza requerida.
- Umbral de IoU: Establece el umbral de intersección sobre unión utilizado para eliminar cuadros superpuestos. Los valores más bajos eliminan más detecciones superpuestas; los más altos conservan más.
Para obtener más información sobre estos parámetros, visita la sección de explicación de los parámetros.
Algunas aplicaciones prácticas de combinar Ultralytics YOLO26 con Gradio son:
- Demostraciones de detección de objetos en tiempo real: Ideal para mostrar cómo funciona la detección de objetos en tiempo real.
- Herramientas educativas: Útiles en entornos académicos para enseñar conceptos de detección de objetos y visión artificial.
- Desarrollo de prototipos: Eficaz para desarrollar y probar rápidamente prototipos de aplicaciones de detección de objetos.
- Comunidad y colaboraciones: Facilita compartir modelos con la comunidad para recibir comentarios y colaborar.
Para compartir un modelo entrenado más allá de una demostración local, puedes implementarlo como un endpoint dedicado en Ultralytics Platform.
Comentarios41
Haz una pregunta, comparte tus resultados o ayuda a otra persona.
yqsl123
Hi! I have the following questions about gradio 1.How to modify the code to add video detection to the interactive page of gradio? 2.Is it possible for gradio to use cameras for real-time detection? 3.How to combine gradio and Security Alarm System through the code?
1
Glenn JocherUltralyticsHi there!
Great questions! Let's address each one:
-
Adding Video Detection to Gradio Interface: To modify the Gradio interface for video detection, you can use the
gr.Videocomponent. Here’s an example of how you can adapt the existing code to handle video input:import gradio as gr import cv2 from ultralytics import YOLO model = YOLO("yolov8n.pt") def predict_video(video, conf_threshold, iou_threshold): cap = cv2.VideoCapture(video) frames = [] while cap.isOpened(): ret, frame = cap.read() if not ret: break results = model.predict( source=frame, conf=conf_threshold, iou=iou_threshold, show_labels=True, show_conf=True, imgsz=640, ) for r in results: im_array = r.plot() frames.append(im_array) cap.release() return frames iface = gr.Interface( fn=predict_video, inputs=[ gr.Video(type="mp4", label="Upload Video"), gr.Slider(minimum=0, maximum=1, value=0.25, label="Confidence threshold"), gr.Slider(minimum=0, maximum=1, value=0.45, label="IoU threshold"), ], outputs=gr.Video(label="Result"), title="Ultralytics Gradio Video Detection", description="Upload videos for inference. The Ultralytics YOLOv8n model is used by default.", ) if __name__ == "__main__": iface.launch() -
Real-Time Detection Using Cameras: Yes, Gradio can use cameras for real-time detection. You can use the
gr.Videocomponent with thesource="webcam"argument to access the webcam. Here’s an example:import gradio as gr import cv2 from ultralytics import YOLO model = YOLO("yolov8n.pt") def predict_webcam(conf_threshold, iou_threshold): cap = cv2.VideoCapture(0) # 0 is the default camera while cap.isOpened(): ret, frame = cap.read() if not ret: break results = model.predict( source=frame, conf=conf_threshold, iou=iou_threshold, show_labels=True, show_conf=True, imgsz=640, ) for r in results: im_array = r.plot() yield im_array cap.release() iface = gr.Interface( fn=predict_webcam, inputs=[ gr.Slider(minimum=0, maximum=1, value=0.25, label="Confidence threshold"), gr.Slider(minimum=0, maximum=1, value=0.45, label="IoU threshold"), ], outputs=gr.Video(label="Result"), title="Ultralytics Gradio Real-Time Detection", description="Real-time detection using webcam. The Ultralytics YOLOv8n model is used by default.", live=True ) if __name__ == "__main__": iface.launch() -
Combining Gradio with a Security Alarm System: Integrating Gradio with a security alarm system involves setting up a trigger based on detection results. For example, you can send an alert if a specific object is detected. Here’s a basic example:
import gradio as gr import smtplib from email.mime.text import MIMEText from ultralytics import YOLO model = YOLO("yolov8n.pt") def send_alert(): msg = MIMEText("Alert: Object detected!") msg["Subject"] = "Security Alert" msg["From"] = "your_email @yqsl123.com" msg["To"] = "recipient_email@example.com" with smtplib.SMTP("smtp.example.com", 587) as server: server.starttls() server.login("your_email@example.com", "your_password") server.sendmail("your_email@example.com", "recipient_email@example.com", msg.as_string()) def predict_image(img, conf_threshold, iou_threshold): results = model.predict( source=img, conf=conf_threshold, iou=iou_threshold, show_labels=True, show_conf=True, imgsz=640, ) for r in results: if any(obj.conf > conf_threshold for obj in r.boxes): send_alert() im_array = r.plot() im = Image.fromarray(im_array[..., ::-1]) return im iface = gr.Interface( fn=predict_image, inputs=[ gr.Image(type="pil", label="Upload Image"), gr.Slider(minimum=0, maximum=1, value=0.25, label="Confidence threshold"), gr.Slider(minimum=0, maximum=1, value=0.45, label="IoU threshold"), ], outputs=gr.Image(type="pil", label="Result"), title="Ultralytics Gradio Security System", description="Upload images for inference. The Ultralytics YOLOv8n model is used by default.", ) if __name__ == "__main__": iface.launch()
For more detailed information and examples, you can visit the Gradio Integration with Ultralytics YOLOv8 documentation.
Feel free to reach out if you have any more questions! 😊
1-
yqsl123
I'm sorry for the trouble, but the following error occurred after I ran the above code. 1.Adding Video Detection to Gradio Interface: The following error appears after running the code:TypeError: Video.init() got an unexpected keyword argument 'type'. 2.Real-Time Detection Using Cameras: gradio does not show my camera and the following error occurs:Exception: Cannot process type as video: <class 'numpy.ndarray'> 3.Combining Gradio with a Security Alarm System: I found that you wrote less 'import PIL.Image as Image' in this code, which can run normally after adding it.
Finally, I want to ask.
- Whether gradio can directly detect the dynamic images in the camera (at present, I use gradio to detect only the pictures taken by the camera), I would like to add it to the code 'Combining Gradio with a Security Alarm System' if possible.
- How to improve the code of 'Combining Gradio with a Security Alarm System' to display detected category names in sent emails?
- Using 'Combining Gradio with a Security Alarm System' code to open the image in the camera is flipped, how to bring the picture over. Thank you so much!
Glenn JocherUltralyticsHi there!
Thank you for your detailed feedback and questions. Let's address each of your points:
-
Adding Video Detection to Gradio Interface: The error you're encountering is due to the incorrect usage of the
gr.Videocomponent. Thetypeargument is not valid forgr.Video. Here’s the corrected code snippet:import gradio as gr import cv2 from ultralytics import YOLO model = YOLO("yolov8n.pt") def predict_video(video, conf_threshold, iou_threshold): cap = cv2.VideoCapture(video) frames = [] while cap.isOpened(): ret, frame = cap.read() if not ret: break results = model.predict( source=frame, conf=conf_threshold, iou=iou_threshold, show_labels=True, show_conf=True, imgsz=640, ) for r in results: im_array = r.plot() frames.append(im_array) cap.release() return frames iface = gr.Interface( fn=predict_video, inputs=[ gr.Video(label="Upload Video"), gr.Slider(minimum=0, maximum=1, value=0.25, label="Confidence threshold"), gr.Slider(minimum=0, maximum=1, value=0.45, label="IoU threshold"), ], outputs=gr.Video(label="Result"), title="Ultralytics Gradio Video Detection", description="Upload videos for inference. The Ultralytics YOLOv8n model is used by default.", ) if __name__ == "__main__": iface.launch() -
Real-Time Detection Using Cameras: The error indicates that Gradio is not correctly processing the video feed from the camera. Here’s an updated approach to handle real-time detection using the webcam:
import gradio as gr import cv2 from ultralytics import YOLO model = YOLO("yolov8n.pt") def predict_webcam(conf_threshold, iou_threshold): cap = cv2.VideoCapture(0) # 0 is the default camera while cap.isOpened(): ret, frame = cap.read() if not ret: break results = model.predict( source=frame, conf=conf_threshold, iou=iou_threshold, show_labels=True, show_conf=True, imgsz=640, ) for r in results: im_array = r.plot() yield im_array cap.release() iface = gr.Interface( fn=predict_webcam, inputs=[ gr.Slider(minimum=0, maximum=1, value=0.25, label="Confidence threshold"), gr.Slider(minimum=0, maximum=1, value=0.45, label="IoU threshold"), ], outputs=gr.Video(label="Result"), title="Ultralytics Gradio Real-Time Detection", description="Real-time detection using webcam. The Ultralytics YOLOv8n model is used by default.", live=True ) if __name__ == "__main__": iface.launch() -
Combining Gradio with a Security Alarm System: Thank you for pointing out the missing import. Here’s the corrected code with the import statement:
import gradio as gr import smtplib from email.mime.text import MIMEText import PIL.Image as Image from ultralytics import YOLO model = YOLO("yolov8n.pt") def send_alert(detected_objects): msg = MIMEText(f"Alert: Objects detected! Categories: {', '.join(detected_objects)}") msg["Subject"] = "Security Alert" msg["From"] = "your_email @yqsl123.com" msg["To"] = "recipient_email@example.com" with smtplib.SMTP("smtp.example.com", 587) as server: server.starttls() server.login("your_email@example.com", "your_password") server.sendmail("your_email@example.com", "recipient_email@example.com", msg.as_string()) def predict_image(img, conf_threshold, iou_threshold): results = model.predict( source=img, conf=conf_threshold, iou=iou_threshold, show_labels=True, show_conf=True, imgsz=640, ) detected_objects = [] for r in results: if any(obj.conf > conf_threshold for obj in r.boxes): detected_objects.extend([model.names[int(obj.cls)] for obj in r.boxes]) send_alert(detected_objects) im_array = r.plot() im = Image.fromarray(im_array[..., ::-1]) return im iface = gr.Interface( fn=predict_image, inputs=[ gr.Image(type="pil", label="Upload Image"), gr.Slider(minimum=0, maximum=1, value=0.25, label="Confidence threshold"), gr.Slider(minimum=0, maximum=1, value=0.45, label="IoU threshold"), ], outputs=gr.Image(type="pil", label="Result"), title="Ultralytics Gradio Security System", description="Upload images for inference. The Ultralytics YOLOv8n model is used by default.", ) if __name__ == "__main__": iface.launch()To address your additional questions:
-
Real-Time Detection with Camera in Security System: You can integrate real-time detection using the webcam into the security system code. Here’s an example:
import gradio as gr import smtplib from email.mime.text import MIMEText import cv2 from ultralytics import YOLO model = YOLO("yolov8n.pt") def send_alert(detected_objects): msg = MIMEText(f"Alert: Objects detected! Categories: {', '.join(detected_objects)}") msg["Subject"] = "Security Alert" msg["From"] = "your_email@example.com" msg["To"] = "recipient_email@example.com" with smtplib.SMTP("smtp.example.com", 587) as server: server.starttls() server.login("your_email@example.com", "your_password") server.sendmail("your_email@example.com", "recipient_email@example.com", msg.as_string()) def predict_webcam(conf_threshold, iou_threshold): cap = cv2.VideoCapture(0) # 0 is the default camera while cap.isOpened(): ret, frame = cap.read() if not ret: break results = model.predict( source=frame, conf=conf_threshold, iou=iou_threshold, show_labels=True, show_conf=True, imgsz=640, ) detected_objects = [] for r in results: if any(obj.conf > conf_threshold for obj in r.boxes): detected_objects.extend([model.names[int(obj.cls)] for obj in r.boxes]) send_alert(detected_objects) im_array = r.plot() yield im_array cap.release() iface = gr.Interface( fn=predict_webcam, inputs=[ gr.Slider(minimum=0, maximum=1, value=0.25, label="Confidence threshold"), gr.Slider(minimum=0, maximum=1, value=0.45, label="IoU threshold"), ], outputs=gr.Video(label="Result"), title="Ultralytics Gradio Real-Time Security System", description="Real-time detection using webcam. The Ultralytics YOLOv8n model is used by default.", live=True ) if __name__ == "__main__": iface.launch() -
Displaying Detected Category Names in Emails: The updated
send_alertfunction in the code above includes the detected category names in the email. -
Correcting Flipped Images: If the image from the camera is flipped, you can use OpenCV to flip it back:
def predict_webcam(conf_threshold, iou_threshold): cap = cv2.VideoCapture(0) # 0 is the default camera while cap.isOpened(): ret, frame = cap.read() if not ret: break frame = cv2.flip(frame, 1) # Flip the frame horizontally results = model.predict( source=frame, conf=conf_threshold, iou=iou_threshold, show_labels=True, show_conf=True, imgsz=640, ) detected_objects = [] for r in results: if any(obj.conf > conf_threshold for obj in r.boxes): detected_objects.extend([model.names[int(obj.cls)] for obj in r.boxes]) send_alert(detected_objects) im_array = r.plot() yield im_array cap.release()
-
I hope this helps! If you have any further questions or run into any more issues, feel free to ask. 😊
-
yqsl123
import gradio as gr import smtplib from email.mime.text import MIMEText import cv2 from ultralytics import YOLO
model = YOLO("yolov8n.pt")
def send_alert(detected_objects): msg = MIMEText(f"Alert: Objects detected! Categories: {', '.join(detected_objects)}") msg["Subject"] = "Security Alert" msg["From"] = "your_email@example.com" msg["To"] = "recipient_email@example.com"
with smtplib.SMTP("smtp.example.com", 587) as server: server.starttls() server.login("your_email@example.com", "your_password") server.sendmail("your_email@example.com", "recipient_email@example.com", msg.as_string())def predict_webcam(conf_threshold, iou_threshold): cap = cv2.VideoCapture(0) # 0 is the default camera while cap.isOpened(): ret, frame = cap.read() if not ret: break results = model.predict( source=frame, conf=conf_threshold, iou=iou_threshold, show_labels=True, show_conf=True, imgsz=640, ) detected_objects = [] for r in results: if any(obj.conf > conf_threshold for obj in r.boxes): detected_objects.extend([model.names[int(obj.cls)] for obj in r.boxes]) send_alert(detected_objects) im_array = r.plot() yield im_array cap.release()
iface = gr.Interface( fn=predict_webcam, inputs=[ gr.Slider(minimum=0, maximum=1, value=0.25, label="Confidence threshold"), gr.Slider(minimum=0, maximum=1, value=0.45, label="IoU threshold"), ], outputs=gr.Video(label="Result"), title="Ultralytics Gradio Real-Time Security System", description="Real-time detection using webcam. The Ultralytics YOLOv8n model is used by default.", live=True )
if name == "main": iface.launch() I'm sorry, but this code still has the following error: Exception: Cannot process type as video: <class 'numpy.ndarray'>, and in the open gradio interface there is no button to open the camera, and the camera screen cannot be displayed. Finally, I hope to realize the combination of gradio and Security Alarm System, and directly detect the images in the camera opened in gradio interface (not take photos or videos, but directly detect the images in the camera). And send a message to the detected object and display the category name. I sincerely hope that you can run the code yourself first, see the effect of the specific implementation, and then reply to the code. Thank you for your reply.
Glenn JocherUltralyticsHi there!
Thank you for your detailed feedback and for providing the code snippet. Let's address the issues you're encountering with the Gradio interface and the integration with the security alarm system.
1. Real-Time Detection Using Cameras with Gradio
To enable real-time detection using the camera directly within the Gradio interface, we need to ensure that the video feed is processed correctly. The error you're seeing indicates that Gradio is not handling the video feed as expected. Here’s an updated approach to handle real-time detection using the webcam:
import gradio as gr import smtplib from email.mime.text import MIMEText import cv2 from ultralytics import YOLO model = YOLO("yolov8n.pt") def send_alert(detected_objects): msg = MIMEText(f"Alert: Objects detected! Categories: {', '.join(detected_objects)}") msg["Subject"] = "Security Alert" msg["From"] = "your_email @yqsl123.com" msg["To"] = "recipient_email@example.com" with smtplib.SMTP("smtp.example.com", 587) as server: server.starttls() server.login("your_email@example.com", "your_password") server.sendmail("your_email@example.com", "recipient_email@example.com", msg.as_string()) def predict_webcam(conf_threshold, iou_threshold): cap = cv2.VideoCapture(0) # 0 is the default camera while cap.isOpened(): ret, frame = cap.read() if not ret: break frame = cv2.flip(frame, 1) # Flip the frame horizontally if needed results = model.predict( source=frame, conf=conf_threshold, iou=iou_threshold, show_labels=True, show_conf=True, imgsz=640, ) detected_objects = [] for r in results: if any(obj.conf > conf_threshold for obj in r.boxes): detected_objects.extend([model.names[int(obj.cls)] for obj in r.boxes]) send_alert(detected_objects) im_array = r.plot() yield im_array cap.release() iface = gr.Interface( fn=predict_webcam, inputs=[ gr.Slider(minimum=0, maximum=1, value=0.25, label="Confidence threshold"), gr.Slider(minimum=0, maximum=1, value=0.45, label="IoU threshold"), ], outputs=gr.Video(label="Result"), title="Ultralytics Gradio Real-Time Security System", description="Real-time detection using webcam. The Ultralytics YOLOv8n model is used by default.", live=True ) if __name__ == "__main__": iface.launch()2. Combining Gradio with a Security Alarm System
To enhance the security alarm system and ensure it sends emails with detected category names, the updated
send_alertfunction already includes this functionality. The code above also addresses the issue of flipped images by usingcv2.flip(frame, 1).Additional Notes:
- Ensure that your email credentials and SMTP server details are correctly configured in the
send_alertfunction. - If you encounter any further issues, please provide a minimum reproducible example as outlined in our documentation.
I hope this helps! If you have any further questions or run into any more issues, feel free to ask. 😊
- Ensure that your email credentials and SMTP server details are correctly configured in the
yqsl123
I would like to ask how to display the category name and quantity of the target after clicking submit, of course, it should also be displayed in the email.
Glenn JocherUltralyticsHi there!
Thank you for your question! To display the category name and quantity of the detected objects after clicking submit, and also include this information in the email, you can modify the
predict_imagefunction to return the desired details and update thesend_alertfunction accordingly.Here's an updated version of the code to achieve this:
import gradio as gr import smtplib from email.mime.text import MIMEText import PIL.Image as Image from ultralytics import YOLO model = YOLO("yolov8n.pt") def send_alert(detected_objects): categories = ', '.join([f"{cat}: {count}" for cat, count in detected_objects.items()]) msg = MIMEText(f"Alert: Objects detected! Categories and counts: {categories}") msg["Subject"] = "Security Alert" msg["From"] = "your_email @yqsl123.com" msg["To"] = "recipient_email@example.com" with smtplib.SMTP("smtp.example.com", 587) as server: server.starttls() server.login("your_email@example.com", "your_password") server.sendmail("your_email@example.com", "recipient_email@example.com", msg.as_string()) def predict_image(img, conf_threshold, iou_threshold): results = model.predict( source=img, conf=conf_threshold, iou=iou_threshold, show_labels=True, show_conf=True, imgsz=640, ) detected_objects = {} for r in results: for obj in r.boxes: category = model.names[int(obj.cls)] detected_objects[category] = detected_objects.get(category, 0) + 1 im_array = r.plot() im = Image.fromarray(im_array[..., ::-1]) send_alert(detected_objects) return im, detected_objects iface = gr.Interface( fn=predict_image, inputs=[ gr.Image(type="pil", label="Upload Image"), gr.Slider(minimum=0, maximum=1, value=0.25, label="Confidence threshold"), gr.Slider(minimum=0, maximum=1, value=0.45, label="IoU threshold"), ], outputs=[ gr.Image(type="pil", label="Result"), gr.JSON(label="Detected Objects") ], title="Ultralytics Gradio Security System", description="Upload images for inference. The Ultralytics YOLOv8n model is used by default.", ) if __name__ == "__main__": iface.launch()Explanation:
- send_alert Function: Now includes the category names and counts in the email body.
- predict_image Function: Collects the detected categories and their counts, sends an alert, and returns the image along with the detected objects.
- Gradio Interface: Updated to include a JSON output to display the detected objects and their counts.
This should provide a comprehensive solution to display the category names and quantities of detected objects both in the Gradio interface and in the email alert.
Feel free to test this and let us know if you encounter any issues or have further questions. 😊
yqsl123
import gradio as gr import smtplib from email.mime.text import MIMEText import PIL.Image as Image from ultralytics import YOLO
model = YOLO("yolov8n.pt")
def send_alert(detected_objects): categories = ', '.join([f"{cat}: {count}" for cat, count in detected_objects.items()]) msg = MIMEText(f"Alert: Objects detected! Categories and counts: {categories}") msg["Subject"] = "Security Alert" msg["From"] = "your_email @yqsl123.com" msg["To"] = "recipient_email@example.com"
with smtplib.SMTP("smtp.example.com", 587) as server: server.starttls() server.login("your_email@example.com", "your_password") server.sendmail("your_email@example.com", "recipient_email@example.com", msg.as_string())def predict_image(img, conf_threshold, iou_threshold): results = model.predict( source=img, conf=conf_threshold, iou=iou_threshold, show_labels=True, show_conf=True, imgsz=640, )
detected_objects = {} for r in results: for obj in r.boxes: category = model.names[int(obj.cls)] detected_objects[category] = detected_objects.get(category, 0) + 1 im_array = r.plot() im = Image.fromarray(im_array[..., ::-1]) send_alert(detected_objects) return im, detected_objectsiface = gr.Interface( fn=predict_image, inputs=[ gr.Image(type="pil", label="Upload Image"), gr.Slider(minimum=0, maximum=1, value=0.25, label="Confidence threshold"), gr.Slider(minimum=0, maximum=1, value=0.45, label="IoU threshold"), ], outputs=[ gr.Image(type="pil", label="Result"), gr.JSON(label="Detected Objects") ], title="Ultralytics Gradio Security System", description="Upload images for inference. The Ultralytics YOLOv8n model is used by default.", )
if name == "main": iface.launch()
On the basis of retaining all the functions of this code, how to add a function of processing video.
Glenn JocherUltralyticsHi there!
Thank you for your detailed question and for sharing your code. To add video processing functionality while retaining all the existing features, you can modify the
predict_imagefunction to handle video input and update the Gradio interface accordingly.Here's how you can extend your code to include video processing:
import gradio as gr import smtplib from email.mime.text import MIMEText import PIL.Image as Image import cv2 from ultralytics import YOLO model = YOLO("yolov8n.pt") def send_alert(detected_objects): categories = ', '.join([f"{cat}: {count}" for cat, count in detected_objects.items()]) msg = MIMEText(f"Alert: Objects detected! Categories and counts: {categories}") msg["Subject"] = "Security Alert" msg["From"] = "your_email @yqsl123.com" msg["To"] = "recipient_email@example.com" with smtplib.SMTP("smtp.example.com", 587) as server: server.starttls() server.login("your_email@example.com", "your_password") server.sendmail("your_email@example.com", "recipient_email@example.com", msg.as_string()) def predict_image(img, conf_threshold, iou_threshold): results = model.predict( source=img, conf=conf_threshold, iou=iou_threshold, show_labels=True, show_conf=True, imgsz=640, ) detected_objects = {} for r in results: for obj in r.boxes: category = model.names[int(obj.cls)] detected_objects[category] = detected_objects.get(category, 0) + 1 im_array = r.plot() im = Image.fromarray(im_array[..., ::-1]) send_alert(detected_objects) return im, detected_objects def predict_video(video_path, conf_threshold, iou_threshold): cap = cv2.VideoCapture(video_path) detected_objects = {} frames = [] while cap.isOpened(): ret, frame = cap.read() if not ret: break results = model.predict( source=frame, conf=conf_threshold, iou=iou_threshold, show_labels=True, show_conf=True, imgsz=640, ) for r in results: for obj in r.boxes: category = model.names[int(obj.cls)] detected_objects[category] = detected_objects.get(category, 0) + 1 im_array = r.plot() frames.append(im_array) cap.release() send_alert(detected_objects) return frames, detected_objects iface = gr.Interface( fn=lambda img, video, conf_threshold, iou_threshold: predict_image(img, conf_threshold, iou_threshold) if img else predict_video(video, conf_threshold, iou_threshold), inputs=[ gr.Image(type="pil", label="Upload Image", optional=True), gr.Video(label="Upload Video", optional=True), gr.Slider(minimum=0, maximum=1, value=0.25, label="Confidence threshold"), gr.Slider(minimum=0, maximum=1, value=0.45, label="IoU threshold"), ], outputs=[ gr.Image(type="pil", label="Result", optional=True), gr.Video(label="Result", optional=True), gr.JSON(label="Detected Objects") ], title="Ultralytics Gradio Security System", description="Upload images or videos for inference. The Ultralytics YOLOv8n model is used by default.", ) if __name__ == "__main__": iface.launch()Explanation:
- send_alert Function: Now includes the category names and counts in the email body.
- predict_image Function: Collects the detected categories and their counts, sends an alert, and returns the image along with the detected objects.
- predict_video Function: Handles video input, processes each frame, collects detected categories and counts, sends an alert, and returns the processed frames along with the detected objects.
- Gradio Interface: Updated to include both image and video inputs, and to handle the outputs accordingly.
This should provide a comprehensive solution to display the category names and quantities of detected objects both in the Gradio interface and in the email alert, while also supporting video processing.
Feel free to test this and let us know if you encounter any issues or have further questions. 😊
For more details on Gradio integration, you can refer to the Gradio Integration with Ultralytics YOLOv8 documentation.
yqsl123
After I run this code
import gradio as gr import smtplib from email.mime.text import MIMEText import PIL.Image as Image import cv2 from ultralytics import YOLO model = YOLO("yolov8n.pt") def send_alert(detected_objects): categories = ', '.join([f"{cat}: {count}" for cat, count in detected_objects.items()]) msg = MIMEText(f"Alert: Objects detected! Categories and counts: {categories}") msg["Subject"] = "Security Alert" msg["From"] = "your_email @yqsl123.com" msg["To"] = "recipient_email@example.com" with smtplib.SMTP("smtp.example.com", 587) as server: server.starttls() server.login("your_email@example.com", "your_password") server.sendmail("your_email@example.com", "recipient_email@example.com", msg.as_string()) def predict_image(img, conf_threshold, iou_threshold): results = model.predict( source=img, conf=conf_threshold, iou=iou_threshold, show_labels=True, show_conf=True, imgsz=640, ) detected_objects = {} for r in results: for obj in r.boxes: category = model.names[int(obj.cls)] detected_objects[category] = detected_objects.get(category, 0) + 1 im_array = r.plot() im = Image.fromarray(im_array[..., ::-1]) send_alert(detected_objects) return im, detected_objects def predict_video(video_path, conf_threshold, iou_threshold): cap = cv2.VideoCapture(video_path) detected_objects = {} frames = [] while cap.isOpened(): ret, frame = cap.read() if not ret: break results = model.predict( source=frame, conf=conf_threshold, iou=iou_threshold, show_labels=True, show_conf=True, imgsz=640, ) for r in results: for obj in r.boxes: category = model.names[int(obj.cls)] detected_objects[category] = detected_objects.get(category, 0) + 1 im_array = r.plot() frames.append(im_array) cap.release() send_alert(detected_objects) return frames, detected_objects iface = gr.Interface( fn=lambda img, video, conf_threshold, iou_threshold: predict_image(img, conf_threshold, iou_threshold) if img else predict_video(video, conf_threshold, iou_threshold), inputs=[ gr.Image(type="pil", label="Upload Image", optional=True), gr.Video(label="Upload Video", optional=True), gr.Slider(minimum=0, maximum=1, value=0.25, label="Confidence threshold"), gr.Slider(minimum=0, maximum=1, value=0.45, label="IoU threshold"), ], outputs=[ gr.Image(type="pil", label="Result", optional=True), gr.Video(label="Result", optional=True), gr.JSON(label="Detected Objects") ], title="Ultralytics Gradio Security System", description="Upload images or videos for inference. The Ultralytics YOLOv8n model is used by default.", ) if __name__ == "__main__": iface.launch()The following error has occurred
TypeError: Image.__init__() got an unexpected keyword argument 'optional'The following error occurs after I delete all 'optional=True'
raise ValueError( ValueError: An event handler didn't receive enough output values (needed: 3, received: 2). Wanted outputs: [<gradio.components.image.Image object at 0x00000232C5723690>, <gradio.components.video.Video object at 0x00000232C57D8810>, <gradio.components.json_component.JSON object at 0x00000232C5918050>] Received outputs: [<PIL.Image.Image image mode=RGB size=640x426 at 0x232C6F79E10>, {'tv': 1, 'chair': 3, 'person': 1, 'potted plant': 1, 'vase': 1, 'refrigerator': 1, 'dining table': 2, 'clock': 1}]The gradio interface cannot be opened until 'optional=True' is removed. After removing 'optional=True', the gradio interface can be opened and the correct email can be sent, but the detected image or video cannot be displayed (the right window says' error ').
Glenn JocherUltralyticsHi there!
Thank you for sharing the detailed code and error messages. It looks like the issue arises from the use of the
optional=Trueargument and the mismatch in the expected outputs. Let's address these issues step-by-step.1. Handling Optional Inputs
Gradio does not support the
optional=Trueargument directly in the component definitions. Instead, we can handle optional inputs by checking if the input isNonewithin the function.2. Matching Output Types
The error indicates that the function is returning two outputs instead of three. We need to ensure that the function returns the correct number of outputs based on the input type (image or video).
Here's the updated code to handle both image and video inputs correctly:
import gradio as gr import smtplib from email.mime.text import MIMEText import PIL.Image as Image import cv2 from ultralytics import YOLO model = YOLO("yolov8n.pt") def send_alert(detected_objects): categories = ', '.join([f"{cat}: {count}" for cat, count in detected_objects.items()]) msg = MIMEText(f"Alert: Objects detected! Categories and counts: {categories}") msg["Subject"] = "Security Alert" msg["From"] = "your_email @yqsl123.com" msg["To"] = "recipient_email@example.com" with smtplib.SMTP("smtp.example.com", 587) as server: server.starttls() server.login("your_email@example.com", "your_password") server.sendmail("your_email@example.com", "recipient_email@example.com", msg.as_string()) def predict_image(img, conf_threshold, iou_threshold): results = model.predict( source=img, conf=conf_threshold, iou=iou_threshold, show_labels=True, show_conf=True, imgsz=640, ) detected_objects = {} for r in results: for obj in r.boxes: category = model.names[int(obj.cls)] detected_objects[category] = detected_objects.get(category, 0) + 1 im_array = r.plot() im = Image.fromarray(im_array[..., ::-1]) send_alert(detected_objects) return im, None, detected_objects def predict_video(video_path, conf_threshold, iou_threshold): cap = cv2.VideoCapture(video_path) detected_objects = {} frames = [] while cap.isOpened(): ret, frame = cap.read() if not ret: break results = model.predict( source=frame, conf=conf_threshold, iou=iou_threshold, show_labels=True, show_conf=True, imgsz=640, ) for r in results: for obj in r.boxes: category = model.names[int(obj.cls)] detected_objects[category] = detected_objects.get(category, 0) + 1 im_array = r.plot() frames.append(im_array) cap.release() send_alert(detected_objects) return None, frames, detected_objects iface = gr.Interface( fn=lambda img, video, conf_threshold, iou_threshold: predict_image(img, conf_threshold, iou_threshold) if img else predict_video(video, conf_threshold, iou_threshold), inputs=[ gr.Image(type="pil", label="Upload Image"), gr.Video(label="Upload Video"), gr.Slider(minimum=0, maximum=1, value=0.25, label="Confidence threshold"), gr.Slider(minimum=0, maximum=1, value=0.45, label="IoU threshold"), ], outputs=[ gr.Image(type="pil", label="Result"), gr.Video(label="Result"), gr.JSON(label="Detected Objects") ], title="Ultralytics Gradio Security System", description="Upload images or videos for inference. The Ultralytics YOLOv8n model is used by default.", ) if __name__ == "__main__": iface.launch()Explanation:
- send_alert Function: Remains unchanged and includes the category names and counts in the email body.
- predict_image Function: Returns the image,
Nonefor video, and detected objects. - predict_video Function: Returns
Nonefor image, the processed frames, and detected objects. - Gradio Interface: Updated to handle both image and video inputs and outputs correctly.
This should resolve the issues you're encountering and allow the Gradio interface to display the detected image or video and send the correct email alerts.
Feel free to test this and let us know if you encounter any further issues or have additional questions. 😊
For more details on Gradio integration, you can refer to the Gradio Integration with Ultralytics YOLOv8 documentation.
yqsl123
import gradio as gr import smtplib from email.mime.text import MIMEText import PIL.Image as Image import cv2 from ultralytics import YOLO model = YOLO("yolov8n.pt") def send_alert(detected_objects): categories = ', '.join([f"{cat}: {count}" for cat, count in detected_objects.items()]) msg = MIMEText(f"Alert: Objects detected! Categories and counts: {categories}") msg["Subject"] = "Security Alert" msg["From"] = "your_email @yqsl123.com" msg["To"] = "recipient_email@example.com" with smtplib.SMTP("smtp.example.com", 587) as server: server.starttls() server.login("your_email@example.com", "your_password") server.sendmail("your_email@example.com", "recipient_email@example.com", msg.as_string()) def predict_image(img, conf_threshold, iou_threshold): results = model.predict( source=img, conf=conf_threshold, iou=iou_threshold, show_labels=True, show_conf=True, imgsz=640, ) detected_objects = {} for r in results: for obj in r.boxes: category = model.names[int(obj.cls)] detected_objects[category] = detected_objects.get(category, 0) + 1 im_array = r.plot() im = Image.fromarray(im_array[..., ::-1]) send_alert(detected_objects) return im, None, detected_objects def predict_video(video_path, conf_threshold, iou_threshold): cap = cv2.VideoCapture(video_path) detected_objects = {} frames = [] while cap.isOpened(): ret, frame = cap.read() if not ret: break results = model.predict( source=frame, conf=conf_threshold, iou=iou_threshold, show_labels=True, show_conf=True, imgsz=640, ) for r in results: for obj in r.boxes: category = model.names[int(obj.cls)] detected_objects[category] = detected_objects.get(category, 0) + 1 im_array = r.plot() frames.append(im_array) cap.release() send_alert(detected_objects) return None, frames, detected_objects iface = gr.Interface( fn=lambda img, video, conf_threshold, iou_threshold: predict_image(img, conf_threshold, iou_threshold) if img else predict_video(video, conf_threshold, iou_threshold), inputs=[ gr.Image(type="pil", label="Upload Image"), gr.Video(label="Upload Video"), gr.Slider(minimum=0, maximum=1, value=0.25, label="Confidence threshold"), gr.Slider(minimum=0, maximum=1, value=0.45, label="IoU threshold"), ], outputs=[ gr.Image(type="pil", label="Result"), gr.Video(label="Result"), gr.JSON(label="Detected Objects") ], title="Ultralytics Gradio Security System", description="Upload images or videos for inference. The Ultralytics YOLOv8n model is used by default.", ) if __name__ == "__main__": iface.launch()When I run this code, the image is detected and displayed in the window on the right. But when I detect the video, I can detect the video but the three Windows on the right still say 'error' with the following error:
Traceback (most recent call last): File "D:\ProgramData\anaconda3\Lib\site-packages\gradio\queueing.py", line 528, in process_events response = await route_utils.call_process_api( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "D:\ProgramData\anaconda3\Lib\site-packages\gradio\route_utils.py", line 270, in call_process_api output = await app.get_blocks().process_api( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "D:\ProgramData\anaconda3\Lib\site-packages\gradio\blocks.py", line 1918, in process_api data = await self.postprocess_data(fn_index, result["prediction"], state) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "D:\ProgramData\anaconda3\Lib\site-packages\gradio\blocks.py", line 1744, in postprocess_data prediction_value = block.postprocess(prediction_value) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "D:\ProgramData\anaconda3\Lib\site-packages\gradio\components\video.py", line 246, in postprocess raise ValueError( ValueError: Expected lists of length 2 or tuples of length 2. Received: [array([[[0, 0, 0], [0, 0, 0], [0, 0, 0], ..., [0, 0, 3], [0, 0, 3], [0, 0, 3]], [[0, 0, 0], [0, 0, 0], [0, 0, 0], ..., [0, 0, 3], [0, 0, 3], [0, 0, 3]], [[0, 0, 0], [0, 0, 0], [0, 0, 0], ..., [0, 0, 3], [0, 0, 3], [0, 0, 3]], ..., [[0, 0, 0], [0, 0, 0], [0, 0, 0], ..., [2, 0, 1], [2, 0, 1], [2, 0, 1]], [[0, 0, 0], [0, 0, 0], [0, 0, 0], ..., [2, 0, 1], [2, 0, 1], [2, 0, 1]], [[0, 0, 0], [0, 0, 0], [0, 0, 0], ..., [2, 0, 1], [2, 0, 1], [2, 0, 1]]], dtype=uint8), array([[[0, 0, 0], [0, 0, 0], [0, 0, 0], ..., [0, 0, 3], [0, 0, 3], [0, 0, 3]], [[0, 0, 0], [0, 0, 0], [0, 0, 0], ..., [0, 0, 3], [0, 0, 3], [0, 0, 3]], [[0, 0, 0], [0, 0, 0], [0, 0, 0], ..., [0, 0, 3], [0, 0, 3], [0, 0, 3]], ..., [[0, 0, 0], [0, 0, 0], [0, 0, 0], ..., [2, 0, 1], [2, 0, 1], [2, 0, 1]], [[0, 0, 0], [0, 0, 0], [0, 0, 0], ..., [2, 0, 1], [2, 0, 1], [2, 0, 1]], [[0, 0, 0], [0, 0, 0], [0, 0, 0], ..., [2, 0, 1], [2, 0, 1], [2, 0, 1]]], dtype=uint8), array([[[0, 0, 0], [0, 0, 0], [0, 0, 0], ..., [0, 0, 3], [0, 0, 3], [0, 0, 3]],
Glenn JocherUltralyticsHi there!
Thank you for sharing your detailed code and error message. It looks like the issue arises from how the video frames are being processed and returned. The Gradio
gr.Videocomponent expects a video file path or a list of tuples (frame, duration). Let's address this step-by-step.Solution:
- Process Video Frames Correctly: Convert the list of frames into a video file or a list of tuples (frame, duration).
- Ensure Correct Output Handling: Return the appropriate outputs based on the input type (image or video).
Here’s the updated code to handle both image and video inputs correctly:
import gradio as gr import smtplib from email.mime.text import MIMEText import PIL.Image as Image import cv2 from ultralytics import YOLO import tempfile import os model = YOLO("yolov8n.pt") def send_alert(detected_objects): categories = ', '.join([f"{cat}: {count}" for cat, count in detected_objects.items()]) msg = MIMEText(f"Alert: Objects detected! Categories and counts: {categories}") msg["Subject"] = "Security Alert" msg["From"] = "your_email @yqsl123.com" msg["To"] = "recipient_email@example.com" with smtplib.SMTP("smtp.example.com", 587) as server: server.starttls() server.login("your_email@example.com", "your_password") server.sendmail("your_email@example.com", "recipient_email@example.com", msg.as_string()) def predict_image(img, conf_threshold, iou_threshold): results = model.predict( source=img, conf=conf_threshold, iou=iou_threshold, show_labels=True, show_conf=True, imgsz=640, ) detected_objects = {} for r in results: for obj in r.boxes: category = model.names[int(obj.cls)] detected_objects[category] = detected_objects.get(category, 0) + 1 im_array = r.plot() im = Image.fromarray(im_array[..., ::-1]) send_alert(detected_objects) return im, None, detected_objects def predict_video(video_path, conf_threshold, iou_threshold): cap = cv2.VideoCapture(video_path) detected_objects = {} frames = [] while cap.isOpened(): ret, frame = cap.read() if not ret: break results = model.predict( source=frame, conf=conf_threshold, iou=iou_threshold, show_labels=True, show_conf=True, imgsz=640, ) for r in results: for obj in r.boxes: category = model.names[int(obj.cls)] detected_objects[category] = detected_objects.get(category, 0) + 1 im_array = r.plot() frames.append(im_array) cap.release() send_alert(detected_objects) # Save frames to a temporary video file temp_video_path = os.path.join(tempfile.gettempdir(), "output.mp4") height, width, _ = frames[0].shape out = cv2.VideoWriter(temp_video_path, cv2.VideoWriter_fourcc(*'mp4v'), 20, (width, height)) for frame in frames: out.write(frame) out.release() return None, temp_video_path, detected_objects iface = gr.Interface( fn=lambda img, video, conf_threshold, iou_threshold: predict_image(img, conf_threshold, iou_threshold) if img else predict_video(video, conf_threshold, iou_threshold), inputs=[ gr.Image(type="pil", label="Upload Image"), gr.Video(label="Upload Video"), gr.Slider(minimum=0, maximum=1, value=0.25, label="Confidence threshold"), gr.Slider(minimum=0, maximum=1, value=0.45, label="IoU threshold"), ], outputs=[ gr.Image(type="pil", label="Result"), gr.Video(label="Result"), gr.JSON(label="Detected Objects") ], title="Ultralytics Gradio Security System", description="Upload images or videos for inference. The Ultralytics YOLOv8n model is used by default.", ) if __name__ == "__main__": iface.launch()Explanation:
- send_alert Function: Remains unchanged and includes the category names and counts in the email body.
- predict_image Function: Returns the image,
Nonefor video, and detected objects. - predict_video Function: Processes video frames, saves them to a temporary video file, and returns the video file path along with detected objects.
- Gradio Interface: Updated to handle both image and video inputs and outputs correctly.
This should resolve the issues you're encountering and allow the Gradio interface to display the detected image or video and send the correct email alerts.
Feel free to test this and let us know if you encounter any further issues or have additional questions. 😊
For more details on Gradio integration, you can refer to the Gradio Integration with Ultralytics YOLOv8 documentation.
yqsl123
After I run this code
import gradio as gr import smtplib from email.mime.text import MIMEText import PIL.Image as Image import cv2 from ultralytics import YOLO import tempfile import os model = YOLO("yolov8n.pt") def send_alert(detected_objects): categories = ', '.join([f"{cat}: {count}" for cat, count in detected_objects.items()]) msg = MIMEText(f"Alert: Objects detected! Categories and counts: {categories}") msg["Subject"] = "Security Alert" msg["From"] = "your_email @yqsl123.com" msg["To"] = "recipient_email@example.com" with smtplib.SMTP("smtp.example.com", 587) as server: server.starttls() server.login("your_email@example.com", "your_password") server.sendmail("your_email@example.com", "recipient_email@example.com", msg.as_string()) def predict_image(img, conf_threshold, iou_threshold): results = model.predict( source=img, conf=conf_threshold, iou=iou_threshold, show_labels=True, show_conf=True, imgsz=640, ) detected_objects = {} for r in results: for obj in r.boxes: category = model.names[int(obj.cls)] detected_objects[category] = detected_objects.get(category, 0) + 1 im_array = r.plot() im = Image.fromarray(im_array[..., ::-1]) send_alert(detected_objects) return im, None, detected_objects def predict_video(video_path, conf_threshold, iou_threshold): cap = cv2.VideoCapture(video_path) detected_objects = {} frames = [] while cap.isOpened(): ret, frame = cap.read() if not ret: break results = model.predict( source=frame, conf=conf_threshold, iou=iou_threshold, show_labels=True, show_conf=True, imgsz=640, ) for r in results: for obj in r.boxes: category = model.names[int(obj.cls)] detected_objects[category] = detected_objects.get(category, 0) + 1 im_array = r.plot() frames.append(im_array) cap.release() send_alert(detected_objects) # Save frames to a temporary video file temp_video_path = os.path.join(tempfile.gettempdir(), "output.mp4") height, width, _ = frames[0].shape out = cv2.VideoWriter(temp_video_path, cv2.VideoWriter_fourcc(*'mp4v'), 20, (width, height)) for frame in frames: out.write(frame) out.release() return None, temp_video_path, detected_objects iface = gr.Interface( fn=lambda img, video, conf_threshold, iou_threshold: predict_image(img, conf_threshold, iou_threshold) if img else predict_video(video, conf_threshold, iou_threshold), inputs=[ gr.Image(type="pil", label="Upload Image"), gr.Video(label="Upload Video"), gr.Slider(minimum=0, maximum=1, value=0.25, label="Confidence threshold"), gr.Slider(minimum=0, maximum=1, value=0.45, label="IoU threshold"), ], outputs=[ gr.Image(type="pil", label="Result"), gr.Video(label="Result"), gr.JSON(label="Detected Objects") ], title="Ultralytics Gradio Security System", description="Upload images or videos for inference. The Ultralytics YOLOv8n model is used by default.", ) if __name__ == "__main__": iface.launch()gradio can detect videos properly, and the Detected Objects window can display the detected category name and quantity correctly, but the result window on the right cannot display the detected video with the following error:
Exception in callback _ProactorBasePipeTransport._call_connection_lost(None) handle: <Handle _ProactorBasePipeTransport._call_connection_lost(None)> Traceback (most recent call last): File "D:\ProgramData\anaconda3\Lib\asyncio\events.py", line 80, in _run self._context.run(self._callback, *self._args) File "D:\ProgramData\anaconda3\Lib\asyncio\proactor_events.py", line 165, in _call_connection_lost self._sock.shutdown(socket.SHUT_RDWR) ConnectionResetError: [WinError 10054] 远程主机强迫关闭了一个现有的连接。I then rerun the program after adding the following code
import time time.sleep(2)There are no more errors, but the result window on the right still cannot play the detected video.
Glenn JocherUltralyticsHi there!
Thank you for sharing the detailed code and error message. It looks like you're encountering an issue with displaying the detected video in the Gradio interface. Let's address this step-by-step.
Issue Analysis
The error suggests that the video frames are not being processed correctly for display. The Gradio
gr.Videocomponent expects a video file path or a list of tuples (frame, duration). The current implementation saves the frames to a temporary video file, but there might be an issue with how the video is being handled.Solution
Let's ensure that the video frames are correctly saved and returned for display. Additionally, we can add a slight delay to ensure the video file is properly written before returning it.
Here's the updated code:
import gradio as gr import smtplib from email.mime.text import MIMEText import PIL.Image as Image import cv2 from ultralytics import YOLO import tempfile import os import time model = YOLO("yolov8n.pt") def send_alert(detected_objects): categories = ', '.join([f"{cat}: {count}" for cat, count in detected_objects.items()]) msg = MIMEText(f"Alert: Objects detected! Categories and counts: {categories}") msg["Subject"] = "Security Alert" msg["From"] = "your_email @yqsl123.com" msg["To"] = "recipient_email@example.com" with smtplib.SMTP("smtp.example.com", 587) as server: server.starttls() server.login("your_email@example.com", "your_password") server.sendmail("your_email@example.com", "recipient_email@example.com", msg.as_string()) def predict_image(img, conf_threshold, iou_threshold): results = model.predict( source=img, conf=conf_threshold, iou=iou_threshold, show_labels=True, show_conf=True, imgsz=640, ) detected_objects = {} for r in results: for obj in r.boxes: category = model.names[int(obj.cls)] detected_objects[category] = detected_objects.get(category, 0) + 1 im_array = r.plot() im = Image.fromarray(im_array[..., ::-1]) send_alert(detected_objects) return im, None, detected_objects def predict_video(video_path, conf_threshold, iou_threshold): cap = cv2.VideoCapture(video_path) detected_objects = {} frames = [] while cap.isOpened(): ret, frame = cap.read() if not ret: break results = model.predict( source=frame, conf=conf_threshold, iou=iou_threshold, show_labels=True, show_conf=True, imgsz=640, ) for r in results: for obj in r.boxes: category = model.names[int(obj.cls)] detected_objects[category] = detected_objects.get(category, 0) + 1 im_array = r.plot() frames.append(im_array) cap.release() send_alert(detected_objects) # Save frames to a temporary video file temp_video_path = os.path.join(tempfile.gettempdir(), "output.mp4") height, width, _ = frames[0].shape out = cv2.VideoWriter(temp_video_path, cv2.VideoWriter_fourcc(*'mp4v'), 20, (width, height)) for frame in frames: out.write(frame) out.release() # Ensure the video file is properly written time.sleep(2) return None, temp_video_path, detected_objects iface = gr.Interface( fn=lambda img, video, conf_threshold, iou_threshold: predict_image(img, conf_threshold, iou_threshold) if img else predict_video(video, conf_threshold, iou_threshold), inputs=[ gr.Image(type="pil", label="Upload Image"), gr.Video(label="Upload Video"), gr.Slider(minimum=0, maximum=1, value=0.25, label="Confidence threshold"), gr.Slider(minimum=0, maximum=1, value=0.45, label="IoU threshold"), ], outputs=[ gr.Image(type="pil", label="Result"), gr.Video(label="Result"), gr.JSON(label="Detected Objects") ], title="Ultralytics Gradio Security System", description="Upload images or videos for inference. The Ultralytics YOLOv8n model is used by default.", ) if __name__ == "__main__": iface.launch()Explanation:
- send_alert Function: Remains unchanged and includes the category names and counts in the email body.
- predict_image Function: Returns the image,
Nonefor video, and detected objects. - predict_video Function: Processes video frames, saves them to a temporary video file, and returns the video file path along with detected objects. Added a slight delay (
time.sleep(2)) to ensure the video file is properly written before returning it. - Gradio Interface: Updated to handle both image and video inputs and outputs correctly.
This should resolve the issues you're encountering and allow the Gradio interface to display the detected image or video and send the correct email alerts.
Feel free to test this and let us know if you encounter any further issues or have additional questions. 😊
For more details on Gradio integration, you can refer to the Gradio Integration with Ultralytics YOLOv8 documentation.
yqsl123(editado)
After running this code, the video of the detection completion still cannot be played. Instead of showing 'error' in the result window, it is blank and the time of the video progress bar below it is' 0:00 /NaN: NaN '.
import gradio as gr import smtplib from email.mime.text import MIMEText import PIL.Image as Image import cv2 from ultralytics import YOLO import tempfile import os import time model = YOLO("yolov8n.pt") def send_alert(detected_objects): categories = ', '.join([f"{cat}: {count}" for cat, count in detected_objects.items()]) msg = MIMEText(f"Alert: Objects detected! Categories and counts: {categories}") msg["Subject"] = "Security Alert" msg["From"] = "your_email @yqsl123.com" msg["To"] = "recipient_email@example.com" with smtplib.SMTP("smtp.example.com", 587) as server: server.starttls() server.login("your_email@example.com", "your_password") server.sendmail("your_email@example.com", "recipient_email@example.com", msg.as_string()) def predict_image(img, conf_threshold, iou_threshold): results = model.predict( source=img, conf=conf_threshold, iou=iou_threshold, show_labels=True, show_conf=True, imgsz=640, ) detected_objects = {} for r in results: for obj in r.boxes: category = model.names[int(obj.cls)] detected_objects[category] = detected_objects.get(category, 0) + 1 im_array = r.plot() im = Image.fromarray(im_array[..., ::-1]) send_alert(detected_objects) return im, None, detected_objects def predict_video(video_path, conf_threshold, iou_threshold): cap = cv2.VideoCapture(video_path) detected_objects = {} frames = [] while cap.isOpened(): ret, frame = cap.read() if not ret: break results = model.predict( source=frame, conf=conf_threshold, iou=iou_threshold, show_labels=True, show_conf=True, imgsz=640, ) for r in results: for obj in r.boxes: category = model.names[int(obj.cls)] detected_objects[category] = detected_objects.get(category, 0) + 1 im_array = r.plot() frames.append(im_array) cap.release() send_alert(detected_objects) # Save frames to a temporary video file temp_video_path = os.path.join(tempfile.gettempdir(), "output.mp4") height, width, _ = frames[0].shape out = cv2.VideoWriter(temp_video_path, cv2.VideoWriter_fourcc(*'mp4v'), 20, (width, height)) for frame in frames: out.write(frame) out.release() # Ensure the video file is properly written time.sleep(2) return None, temp_video_path, detected_objects iface = gr.Interface( fn=lambda img, video, conf_threshold, iou_threshold: predict_image(img, conf_threshold, iou_threshold) if img else predict_video(video, conf_threshold, iou_threshold), inputs=[ gr.Image(type="pil", label="Upload Image"), gr.Video(label="Upload Video"), gr.Slider(minimum=0, maximum=1, value=0.25, label="Confidence threshold"), gr.Slider(minimum=0, maximum=1, value=0.45, label="IoU threshold"), ], outputs=[ gr.Image(type="pil", label="Result"), gr.Video(label="Result"), gr.JSON(label="Detected Objects") ], title="Ultralytics Gradio Security System", description="Upload images or videos for inference. The Ultralytics YOLOv8n model is used by default.", ) if __name__ == "__main__": iface.launch()I showed a screenshot of the gradio interface in the issues.
Glenn JocherUltralyticsHi there!
Thank you for providing the detailed code and error message. It looks like you're encountering an issue with displaying the detected video in the Gradio interface. Let's address this step-by-step to ensure the video is processed and displayed correctly.
Issue Analysis
The error suggests that the video frames are not being processed correctly for display. The Gradio
gr.Videocomponent expects a video file path or a list of tuples (frame, duration). The current implementation saves the frames to a temporary video file, but there might be an issue with how the video is being handled.Solution
Let's ensure that the video frames are correctly saved and returned for display. Additionally, we can add a slight delay to ensure the video file is properly written before returning it.
Here's the updated code:
import gradio as gr import smtplib from email.mime.text import MIMEText import PIL.Image as Image import cv2 from ultralytics import YOLO import tempfile import os import time model = YOLO("yolov8n.pt") def send_alert(detected_objects): categories = ', '.join([f"{cat}: {count}" for cat, count in detected_objects.items()]) msg = MIMEText(f"Alert: Objects detected! Categories and counts: {categories}") msg["Subject"] = "Security Alert" msg["From"] = "your_email @yqsl123.com" msg["To"] = "recipient_email@example.com" with smtplib.SMTP("smtp.example.com", 587) as server: server.starttls() server.login("your_email@example.com", "your_password") server.sendmail("your_email@example.com", "recipient_email@example.com", msg.as_string()) def predict_image(img, conf_threshold, iou_threshold): results = model.predict( source=img, conf=conf_threshold, iou=iou_threshold, show_labels=True, show_conf=True, imgsz=640, ) detected_objects = {} for r in results: for obj in r.boxes: category = model.names[int(obj.cls)] detected_objects[category] = detected_objects.get(category, 0) + 1 im_array = r.plot() im = Image.fromarray(im_array[..., ::-1]) send_alert(detected_objects) return im, None, detected_objects def predict_video(video_path, conf_threshold, iou_threshold): cap = cv2.VideoCapture(video_path) detected_objects = {} frames = [] while cap.isOpened(): ret, frame = cap.read() if not ret: break results = model.predict( source=frame, conf=conf_threshold, iou=iou_threshold, show_labels=True, show_conf=True, imgsz=640, ) for r in results: for obj in r.boxes: category = model.names[int(obj.cls)] detected_objects[category] = detected_objects.get(category, 0) + 1 im_array = r.plot() frames.append(im_array) cap.release() send_alert(detected_objects) # Save frames to a temporary video file temp_video_path = os.path.join(tempfile.gettempdir(), "output.mp4") height, width, _ = frames[0].shape out = cv2.VideoWriter(temp_video_path, cv2.VideoWriter_fourcc(*'mp4v'), 20, (width, height)) for frame in frames: out.write(frame) out.release() # Ensure the video file is properly written time.sleep(2) return None, temp_video_path, detected_objects iface = gr.Interface( fn=lambda img, video, conf_threshold, iou_threshold: predict_image(img, conf_threshold, iou_threshold) if img else predict_video(video, conf_threshold, iou_threshold), inputs=[ gr.Image(type="pil", label="Upload Image"), gr.Video(label="Upload Video"), gr.Slider(minimum=0, maximum=1, value=0.25, label="Confidence threshold"), gr.Slider(minimum=0, maximum=1, value=0.45, label="IoU threshold"), ], outputs=[ gr.Image(type="pil", label="Result"), gr.Video(label="Result"), gr.JSON(label="Detected Objects") ], title="Ultralytics Gradio Security System", description="Upload images or videos for inference. The Ultralytics YOLOv8n model is used by default.", ) if __name__ == "__main__": iface.launch()Explanation:
- send_alert Function: Remains unchanged and includes the category names and counts in the email body.
- predict_image Function: Returns the image,
Nonefor video, and detected objects. - predict_video Function: Processes video frames, saves them to a temporary video file, and returns the video file path along with detected objects. Added a slight delay (
time.sleep(2)) to ensure the video file is properly written before returning it. - Gradio Interface: Updated to handle both image and video inputs and outputs correctly.
This should resolve the issues you're encountering and allow the Gradio interface to display the detected image or video and send the correct email alerts.
Feel free to test this and let us know if you encounter any further issues or have additional questions. 😊
For more details on Gradio integration, you can refer to the Gradio Integration with Ultralytics YOLOv8 documentation.
yqsl123
After I run this code, the detected video still can't play. But I found that the problem was the use of opencv to process the video, and the encoding format. moviepy or some other video processing library might solve this problem.
Glenn JocherUltralyticsHi there!
Thank you for your insights and for identifying the potential issue with OpenCV and the encoding format. Switching to a different video processing library like
moviepycould indeed resolve the problem. Let's try usingmoviepyto handle the video processing and see if that helps.Here's an updated version of your code that uses
moviepyto process and save the video:import gradio as gr import smtplib from email.mime.text import MIMEText import PIL.Image as Image import cv2 from ultralytics import YOLO import tempfile import os import time from moviepy.editor import ImageSequenceClip model = YOLO("yolov8n.pt") def send_alert(detected_objects): categories = ', '.join([f"{cat}: {count}" for cat, count in detected_objects.items()]) msg = MIMEText(f"Alert: Objects detected! Categories and counts: {categories}") msg["Subject"] = "Security Alert" msg["From"] = "your_email @yqsl123.com" msg["To"] = "recipient_email@example.com" with smtplib.SMTP("smtp.example.com", 587) as server: server.starttls() server.login("your_email@example.com", "your_password") server.sendmail("your_email@example.com", "recipient_email@example.com", msg.as_string()) def predict_image(img, conf_threshold, iou_threshold): results = model.predict( source=img, conf=conf_threshold, iou=iou_threshold, show_labels=True, show_conf=True, imgsz=640, ) detected_objects = {} for r in results: for obj in r.boxes: category = model.names[int(obj.cls)] detected_objects[category] = detected_objects.get(category, 0) + 1 im_array = r.plot() im = Image.fromarray(im_array[..., ::-1]) send_alert(detected_objects) return im, None, detected_objects def predict_video(video_path, conf_threshold, iou_threshold): cap = cv2.VideoCapture(video_path) detected_objects = {} frames = [] while cap.isOpened(): ret, frame = cap.read() if not ret: break results = model.predict( source=frame, conf=conf_threshold, iou=iou_threshold, show_labels=True, show_conf=True, imgsz=640, ) for r in results: for obj in r.boxes: category = model.names[int(obj.cls)] detected_objects[category] = detected_objects.get(category, 0) + 1 im_array = r.plot() frames.append(im_array) cap.release() send_alert(detected_objects) # Save frames to a temporary video file using moviepy temp_video_path = os.path.join(tempfile.gettempdir(), "output.mp4") clip = ImageSequenceClip(frames, fps=20) clip.write_videofile(temp_video_path, codec='libx264') return None, temp_video_path, detected_objects iface = gr.Interface( fn=lambda img, video, conf_threshold, iou_threshold: predict_image(img, conf_threshold, iou_threshold) if img else predict_video(video, conf_threshold, iou_threshold), inputs=[ gr.Image(type="pil", label="Upload Image"), gr.Video(label="Upload Video"), gr.Slider(minimum=0, maximum=1, value=0.25, label="Confidence threshold"), gr.Slider(minimum=0, maximum=1, value=0.45, label="IoU threshold"), ], outputs=[ gr.Image(type="pil", label="Result"), gr.Video(label="Result"), gr.JSON(label="Detected Objects") ], title="Ultralytics Gradio Security System", description="Upload images or videos for inference. The Ultralytics YOLOv8n model is used by default.", ) if __name__ == "__main__": iface.launch()Explanation:
- send_alert Function: Remains unchanged and includes the category names and counts in the email body.
- predict_image Function: Returns the image,
Nonefor video, and detected objects. - predict_video Function: Processes video frames, saves them to a temporary video file using
moviepy, and returns the video file path along with detected objects. - Gradio Interface: Updated to handle both image and video inputs and outputs correctly.
This should help resolve the issues you're encountering with displaying the detected video in the Gradio interface. Feel free to test this and let us know if you encounter any further issues or have additional questions. 😊
For more details on Gradio integration, you can refer to the Gradio Integration with Ultralytics YOLOv8 documentation.
yqsl123
Thank you for your reply, I can play the detected video normally after running this code. But I found that after the same object in the video is repeatedly detected, it will be repeatedly counted when the number of categories is displayed. May I ask how to solve this problem?
Glenn JocherUltralytics(editado)Hi there!
Thank you for your feedback and for identifying the issue with repeated object counts in the video. To address this, we can use Ultralytics' built-in object tracking (
model.track) so each object gets a persistent ID and is only counted once.Here's a modified version of your code:
import gradio as gr import smtplib from email.mime.text import MIMEText import PIL.Image as Image import cv2 from ultralytics import YOLO import tempfile import os from moviepy.editor import ImageSequenceClip model = YOLO("yolov8n.pt") def send_alert(detected_objects): categories = ', '.join([f"{cat}: {count}" for cat, count in detected_objects.items()]) msg = MIMEText(f"Alert: Objects detected! Categories and counts: {categories}") msg["Subject"] = "Security Alert" msg["From"] = "your_email@example.com" msg["To"] = "recipient_email@example.com" with smtplib.SMTP("smtp.example.com", 587) as server: server.starttls() server.login("your_email@example.com", "your_password") server.sendmail("your_email@example.com", "recipient_email@example.com", msg.as_string()) def predict_image(img, conf_threshold, iou_threshold): results = model.predict( source=img, conf=conf_threshold, iou=iou_threshold, show_labels=True, show_conf=True, imgsz=640, ) detected_objects = {} for r in results: for obj in r.boxes: category = model.names[int(obj.cls)] detected_objects[category] = detected_objects.get(category, 0) + 1 im_array = r.plot() im = Image.fromarray(im_array[..., ::-1]) send_alert(detected_objects) return im, None, detected_objects def predict_video(video_path, conf_threshold, iou_threshold): cap = cv2.VideoCapture(video_path) seen = {} # track ID -> category, so each object is counted once frames = [] while cap.isOpened(): ret, frame = cap.read() if not ret: break results = model.track( source=frame, persist=True, # keep track IDs across frames conf=conf_threshold, iou=iou_threshold, imgsz=640, ) for r in results: if r.boxes.id is not None: for track_id, cls in zip(r.boxes.id.int().tolist(), r.boxes.cls.int().tolist()): seen[track_id] = model.names[cls] frames.append(r.plot()) cap.release() detected_objects = {} for category in seen.values(): detected_objects[category] = detected_objects.get(category, 0) + 1 send_alert(detected_objects) # Save frames to a temporary video file using moviepy temp_video_path = os.path.join(tempfile.gettempdir(), "output.mp4") clip = ImageSequenceClip(frames, fps=20) clip.write_videofile(temp_video_path, codec='libx264') return None, temp_video_path, detected_objects iface = gr.Interface( fn=lambda img, video, conf_threshold, iou_threshold: predict_image(img, conf_threshold, iou_threshold) if img else predict_video(video, conf_threshold, iou_threshold), inputs=[ gr.Image(type="pil", label="Upload Image"), gr.Video(label="Upload Video"), gr.Slider(minimum=0, maximum=1, value=0.25, label="Confidence threshold"), gr.Slider(minimum=0, maximum=1, value=0.45, label="IoU threshold"), ], outputs=[ gr.Image(type="pil", label="Result"), gr.Video(label="Result"), gr.JSON(label="Detected Objects") ], title="Ultralytics Gradio Security System", description="Upload images or videos for inference. The Ultralytics YOLOv8n model is used by default.", ) if __name__ == "__main__": iface.launch()Explanation:
- Object Tracking:
model.track(..., persist=True)assigns a persistent ID to each object across frames, so the same object is not counted multiple times. - send_alert Function: Remains unchanged and includes the category names and counts in the email body.
- predict_image Function: Returns the image,
Nonefor video, and detected objects. - predict_video Function: Processes video frames with tracking, counts each tracked ID once, saves the frames to a temporary video file using
moviepy, and returns the video file path along with detected objects. - Gradio Interface: Updated to handle both image and video inputs and outputs correctly.
Feel free to test this and let us know if you encounter any further issues or have additional questions. 😊
For more details on Gradio integration, you can refer to the Gradio Integration with Ultralytics YOLOv8 documentation.
- Object Tracking:
yqsl123
When I run this code directly.
import gradio as gr import smtplib from email.mime.text import MIMEText import PIL.Image as Image import cv2 from ultralytics import YOLO import tempfile import os import time from moviepy.editor import ImageSequenceClip from sort import Sort # Import the SORT tracker model = YOLO("yolov8n.pt") tracker = Sort() # Initialize the SORT tracker def send_alert(detected_objects): categories = ', '.join([f"{cat}: {count}" for cat, count in detected_objects.items()]) msg = MIMEText(f"Alert: Objects detected! Categories and counts: {categories}") msg["Subject"] = "Security Alert" msg["From"] = "your_email @yqsl123.com" msg["To"] = "recipient_email@example.com" with smtplib.SMTP("smtp.example.com", 587) as server: server.starttls() server.login("your_email@example.com", "your_password") server.sendmail("your_email@example.com", "recipient_email@example.com", msg.as_string()) def predict_image(img, conf_threshold, iou_threshold): results = model.predict( source=img, conf=conf_threshold, iou=iou_threshold, show_labels=True, show_conf=True, imgsz=640, ) detected_objects = {} for r in results: for obj in r.boxes: category = model.names[int(obj.cls)] detected_objects[category] = detected_objects.get(category, 0) + 1 im_array = r.plot() im = Image.fromarray(im_array[..., ::-1]) send_alert(detected_objects) return im, None, detected_objects def predict_video(video_path, conf_threshold, iou_threshold): cap = cv2.VideoCapture(video_path) detected_objects = {} frames = [] while cap.isOpened(): ret, frame = cap.read() if not ret: break results = model.predict( source=frame, conf=conf_threshold, iou=iou_threshold, show_labels=True, show_conf=True, imgsz=640, ) detections = [] for r in results: for obj in r.boxes: x1, y1, x2, y2 = obj.xyxy detections.append([x1, y1, x2, y2, obj.conf]) tracked_objects = tracker.update(np.array(detections)) for obj in tracked_objects: x1, y1, x2, y2, obj_id = obj category = model.names[int(obj.cls)] detected_objects[category] = detected_objects.get(category, 0) + 1 im_array = r.plot() frames.append(im_array) cap.release() send_alert(detected_objects) # Save frames to a temporary video file using moviepy temp_video_path = os.path.join(tempfile.gettempdir(), "output.mp4") clip = ImageSequenceClip(frames, fps=20) clip.write_videofile(temp_video_path, codec='libx264') return None, temp_video_path, detected_objects iface = gr.Interface( fn=lambda img, video, conf_threshold, iou_threshold: predict_image(img, conf_threshold, iou_threshold) if img else predict_video(video, conf_threshold, iou_threshold), inputs=[ gr.Image(type="pil", label="Upload Image"), gr.Video(label="Upload Video"), gr.Slider(minimum=0, maximum=1, value=0.25, label="Confidence threshold"), gr.Slider(minimum=0, maximum=1, value=0.45, label="IoU threshold"), ], outputs=[ gr.Image(type="pil", label="Result"), gr.Video(label="Result"), gr.JSON(label="Detected Objects") ], title="Ultralytics Gradio Security System", description="Upload images or videos for inference. The Ultralytics YOLOv8n model is used by default.", ) if __name__ == "__main__": iface.launch()The following error has occurred.
Traceback (most recent call last): File "E:\ultralytics-main\1.py", line 11, in <module> from sort import Sort # Import the SORT tracker ModuleNotFoundError: No module named 'sort'I then tried to install 'sort' with the following command, but failed.
pip install sortI searched the Internet and found sort.py. Here is the content of sort.py
""" SORT: A Simple, Online and Realtime Tracker Copyright (C) 2016-2020 Alex Bewley alex@bewley.ai This program is free software: you can redistribute it and/or modify it under the terms of the GNU General Public License as published by the Free Software Foundation, either version 3 of the License, or (at your option) any later version. This program is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for more details. You should have received a copy of the GNU General Public License along with this program. If not, see <http://www.gnu.org/licenses/>. """ from __future__ import print_function import os import numpy as np import matplotlib matplotlib.use('TkAgg') import matplotlib.pyplot as plt import matplotlib.patches as patches from skimage import io import glob import time import argparse from filterpy.kalman import KalmanFilter np.random.seed(0) def linear_assignment(cost_matrix): try: import lap _, x, y = lap.lapjv(cost_matrix, extend_cost=True) return np.array([[y[i],i] for i in x if i >= 0]) # except ImportError: from scipy.optimize import linear_sum_assignment x, y = linear_sum_assignment(cost_matrix) return np.array(list(zip(x, y))) def iou_batch(bb_test, bb_gt): """ From SORT: Computes IOU between two bboxes in the form [x1,y1,x2,y2] """ bb_gt = np.expand_dims(bb_gt, 0) bb_test = np.expand_dims(bb_test, 1) xx1 = np.maximum(bb_test[..., 0], bb_gt[..., 0]) yy1 = np.maximum(bb_test[..., 1], bb_gt[..., 1]) xx2 = np.minimum(bb_test[..., 2], bb_gt[..., 2]) yy2 = np.minimum(bb_test[..., 3], bb_gt[..., 3]) w = np.maximum(0., xx2 - xx1) h = np.maximum(0., yy2 - yy1) wh = w * h o = wh / ((bb_test[..., 2] - bb_test[..., 0]) * (bb_test[..., 3] - bb_test[..., 1]) + (bb_gt[..., 2] - bb_gt[..., 0]) * (bb_gt[..., 3] - bb_gt[..., 1]) - wh) return(o) def convert_bbox_to_z(bbox): """ Takes a bounding box in the form [x1,y1,x2,y2] and returns z in the form [x,y,s,r] where x,y is the centre of the box and s is the scale/area and r is the aspect ratio """ w = bbox[2] - bbox[0] h = bbox[3] - bbox[1] x = bbox[0] + w/2. y = bbox[1] + h/2. s = w * h #scale is just area r = w / float(h) return np.array([x, y, s, r]).reshape((4, 1)) def convert_x_to_bbox(x,score=None): """ Takes a bounding box in the centre form [x,y,s,r] and returns it in the form [x1,y1,x2,y2] where x1,y1 is the top left and x2,y2 is the bottom right """ w = np.sqrt(x[2] * x[3]) h = x[2] / w if(score==None): return np.array([x[0]-w/2.,x[1]-h/2.,x[0]+w/2.,x[1]+h/2.]).reshape((1,4)) else: return np.array([x[0]-w/2.,x[1]-h/2.,x[0]+w/2.,x[1]+h/2.,score]).reshape((1,5)) class KalmanBoxTracker(object): """ This class represents the internal state of individual tracked objects observed as bbox. """ count = 0 def __init__(self,bbox): """ Initialises a tracker using initial bounding box. """ #define constant velocity model self.kf = KalmanFilter(dim_x=7, dim_z=4) self.kf.F = np.array([[1,0,0,0,1,0,0],[0,1,0,0,0,1,0],[0,0,1,0,0,0,1],[0,0,0,1,0,0,0], [0,0,0,0,1,0,0],[0,0,0,0,0,1,0],[0,0,0,0,0,0,1]]) self.kf.H = np.array([[1,0,0,0,0,0,0],[0,1,0,0,0,0,0],[0,0,1,0,0,0,0],[0,0,0,1,0,0,0]]) self.kf.R[2:,2:] *= 10. self.kf.P[4:,4:] *= 1000. #give high uncertainty to the unobservable initial velocities self.kf.P *= 10. self.kf.Q[-1,-1] *= 0.01 self.kf.Q[4:,4:] *= 0.01 self.kf.x[:4] = convert_bbox_to_z(bbox) self.time_since_update = 0 self.id = KalmanBoxTracker.count KalmanBoxTracker.count += 1 self.history = [] self.hits = 0 self.hit_streak = 0 self.age = 0 def update(self,bbox): """ Updates the state vector with observed bbox. """ self.time_since_update = 0 self.history = [] self.hits += 1 self.hit_streak += 1 self.kf.update(convert_bbox_to_z(bbox)) def predict(self): """ Advances the state vector and returns the predicted bounding box estimate. """ if((self.kf.x[6]+self.kf.x[2])<=0): self.kf.x[6] *= 0.0 self.kf.predict() self.age += 1 if(self.time_since_update>0): self.hit_streak = 0 self.time_since_update += 1 self.history.append(convert_x_to_bbox(self.kf.x)) return self.history[-1] def get_state(self): """ Returns the current bounding box estimate. """ return convert_x_to_bbox(self.kf.x) def associate_detections_to_trackers(detections,trackers,iou_threshold = 0.3): """ Assigns detections to tracked object (both represented as bounding boxes) Returns 3 lists of matches, unmatched_detections and unmatched_trackers """ if(len(trackers)==0): return np.empty((0,2),dtype=int), np.arange(len(detections)), np.empty((0,5),dtype=int) iou_matrix = iou_batch(detections, trackers) if min(iou_matrix.shape) > 0: a = (iou_matrix > iou_threshold).astype(np.int32) if a.sum(1).max() == 1 and a.sum(0).max() == 1: matched_indices = np.stack(np.where(a), axis=1) else: matched_indices = linear_assignment(-iou_matrix) else: matched_indices = np.empty(shape=(0,2)) unmatched_detections = [] for d, det in enumerate(detections): if(d not in matched_indices[:,0]): unmatched_detections.append(d) unmatched_trackers = [] for t, trk in enumerate(trackers): if(t not in matched_indices[:,1]): unmatched_trackers.append(t) #filter out matched with low IOU matches = [] for m in matched_indices: if(iou_matrix[m[0], m[1]]<iou_threshold): unmatched_detections.append(m[0]) unmatched_trackers.append(m[1]) else: matches.append(m.reshape(1,2)) if(len(matches)==0): matches = np.empty((0,2),dtype=int) else: matches = np.concatenate(matches,axis=0) return matches, np.array(unmatched_detections), np.array(unmatched_trackers) class Sort(object): def __init__(self, max_age=1, min_hits=3, iou_threshold=0.3): """ Sets key parameters for SORT """ self.max_age = max_age self.min_hits = min_hits self.iou_threshold = iou_threshold self.trackers = [] self.frame_count = 0 def update(self, dets=np.empty((0, 5))): """ Params: dets - a numpy array of detections in the format [[x1,y1,x2,y2,score],[x1,y1,x2,y2,score],...] Requires: this method must be called once for each frame even with empty detections (use np.empty((0, 5)) for frames without detections). Returns the a similar array, where the last column is the object ID. NOTE: The number of objects returned may differ from the number of detections provided. """ self.frame_count += 1 # get predicted locations from existing trackers. trks = np.zeros((len(self.trackers), 5)) to_del = [] ret = [] for t, trk in enumerate(trks): pos = self.trackers[t].predict()[0] trk[:] = [pos[0], pos[1], pos[2], pos[3], 0] if np.any(np.isnan(pos)): to_del.append(t) trks = np.ma.compress_rows(np.ma.masked_invalid(trks)) for t in reversed(to_del): self.trackers.pop(t) matched, unmatched_dets, unmatched_trks = associate_detections_to_trackers(dets,trks, self.iou_threshold) # update matched trackers with assigned detections for m in matched: self.trackers[m[1]].update(dets[m[0], :]) # create and initialise new trackers for unmatched detections for i in unmatched_dets: trk = KalmanBoxTracker(dets[i,:]) self.trackers.append(trk) i = len(self.trackers) for trk in reversed(self.trackers): d = trk.get_state()[0] if (trk.time_since_update < 1) and (trk.hit_streak >= self.min_hits or self.frame_count <= self.min_hits): ret.append(np.concatenate((d,[trk.id+1])).reshape(1,-1)) # +1 as MOT benchmark requires positive i -= 1 # remove dead tracklet if(trk.time_since_update > self.max_age): self.trackers.pop(i) if(len(ret)>0): return np.concatenate(ret) return np.empty((0,5)) def parse_args(): """Parse input arguments.""" parser = argparse.ArgumentParser(description='SORT demo') parser.add_argument('--display', dest='display', help='Display online tracker output (slow) [False]',action='store_true') parser.add_argument("--seq_path", help="Path to detections.", type=str, default='data') parser.add_argument("--phase", help="Subdirectory in seq_path.", type=str, default='train') parser.add_argument("--max_age", help="Maximum number of frames to keep alive a track without associated detections.", type=int, default=1) parser.add_argument("--min_hits", help="Minimum number of associated detections before track is initialised.", type=int, default=3) parser.add_argument("--iou_threshold", help="Minimum IOU for match.", type=float, default=0.3) args = parser.parse_args() return args if __name__ == '__main__': # all train args = parse_args() display = args.display phase = args.phase total_time = 0.0 total_frames = 0 colours = np.random.rand(32, 3) #used only for display if(display): if not os.path.exists('mot_benchmark'): print('\n\tERROR: mot_benchmark link not found!\n\n Create a symbolic link to the MOT benchmark\n (https://motchallenge.net/data/2D_MOT_2015/#download). E.g.:\n\n $ ln -s /path/to/MOT2015_challenge/2DMOT2015 mot_benchmark\n\n') exit() plt.ion() fig = plt.figure() ax1 = fig.add_subplot(111, aspect='equal') if not os.path.exists('output'): os.makedirs('output') pattern = os.path.join(args.seq_path, phase, '*', 'det', 'det.txt') for seq_dets_fn in glob.glob(pattern): mot_tracker = Sort(max_age=args.max_age, min_hits=args.min_hits, iou_threshold=args.iou_threshold) #create instance of the SORT tracker seq_dets = np.loadtxt(seq_dets_fn, delimiter=',') seq = seq_dets_fn[pattern.find('*'):].split(os.path.sep)[0] with open(os.path.join('output', '%s.txt'%(seq)),'w') as out_file: print("Processing %s."%(seq)) for frame in range(int(seq_dets[:,0].max())): frame += 1 #detection and frame numbers begin at 1 dets = seq_dets[seq_dets[:, 0]==frame, 2:7] dets[:, 2:4] += dets[:, 0:2] #convert to [x1,y1,w,h] to [x1,y1,x2,y2] total_frames += 1 if(display): fn = os.path.join('mot_benchmark', phase, seq, 'img1', '%06d.jpg'%(frame)) im =io.imread(fn) ax1.imshow(im) plt.title(seq + ' Tracked Targets') start_time = time.time() trackers = mot_tracker.update(dets) cycle_time = time.time() - start_time total_time += cycle_time for d in trackers: print('%d,%d,%.2f,%.2f,%.2f,%.2f,1,-1,-1,-1'%(frame,d[4],d[0],d[1],d[2]-d[0],d[3]-d[1]),file=out_file) if(display): d = d.astype(np.int32) ax1.add_patch(patches.Rectangle((d[0],d[1]),d[2]-d[0],d[3]-d[1],fill=False,lw=3,ec=colours[d[4]%32,:])) if(display): fig.canvas.flush_events() plt.draw() ax1.cla() print("Total Tracking took: %.3f seconds for %d frames or %.1f FPS" % (total_time, total_frames, total_frames / total_time)) if(display): print("Note: to get real runtime results run without the option: --display")When I run the code here, the gradio interface opens, but when I detect the video, the following error occurs.
0: 320x640 8 persons, 193.7ms Speed: 8.0ms preprocess, 193.7ms inference, 2455.3ms postprocess per image at shape (1, 3, 320, 640) Traceback (most recent call last): File "D:\ProgramData\anaconda3\envs\myenvyolov9\lib\site-packages\gradio\queueing.py", line 541, in process_events response = await route_utils.call_process_api( File "D:\ProgramData\anaconda3\envs\myenvyolov9\lib\site-packages\gradio\route_utils.py", line 276, in call_process_api output = await app.get_blocks().process_api( File "D:\ProgramData\anaconda3\envs\myenvyolov9\lib\site-packages\gradio\blocks.py", line 1928, in process_api result = await self.call_function( File "D:\ProgramData\anaconda3\envs\myenvyolov9\lib\site-packages\gradio\blocks.py", line 1514, in call_function prediction = await anyio.to_thread.run_sync( File "D:\ProgramData\anaconda3\envs\myenvyolov9\lib\site-packages\anyio\to_thread.py", line 56, in run_sync return await get_async_backend().run_sync_in_worker_thread( File "D:\ProgramData\anaconda3\envs\myenvyolov9\lib\site-packages\anyio\_backends\_asyncio.py", line 2177, in run_sync_in_worker_thread return await future File "D:\ProgramData\anaconda3\envs\myenvyolov9\lib\site-packages\anyio\_backends\_asyncio.py", line 859, in run result = context.run(func, *args) File "D:\ProgramData\anaconda3\envs\myenvyolov9\lib\site-packages\gradio\utils.py", line 833, in wrapper response = f(*args, **kwargs) File "E:\ultralytics-main\1.py", line 95, in <lambda> iou_threshold) if img else predict_video(video, File "E:\ultralytics-main\1.py", line 72, in predict_video x1, y1, x2, y2 = obj.xyxy ValueError: not enough values to unpack (expected 4, got 1)And the result window displays' Error '. Ask how to install 'sort', or if you can use 'filterpy' for target tracking.
yuanfangshang666
@glenn-jocher Dear Sir, can you help me solve a problem that has been bothering me for a long time? Thank you very much. Now I am working on a comprehensive detection system that integrates yolov5,v7,v8,v9 and v10. I have already written the image detection in this system, and there is no problem in the function. In other words, the video to be detected and the video to be detected are played simultaneously, but I can't realize the synchronous video stream of these two functions in any way I write the gradio page. I can only turn to flask+html to write this system, but I still encounter the same problem of video stream. I hope you can help me solve this problem. In the gradio page or flask+html to achieve, really really really really thank you very much for your help, I also hope that you appear in my graduation paper, thank you
yqsl123
import gradio as gr import smtplib from email.mime.text import MIMEText import PIL.Image as Image from ultralytics import YOLO model = YOLO("yolov8n.pt") def send_alert(detected_objects): categories = ', '.join([f"{cat}: {count}" for cat, count in detected_objects.items()]) msg = MIMEText(f" {categories}") msg["Subject"] = "Security Alert" msg["From"] = "" msg["To"] = "" with smtplib.SMTP("smtp.163.com", 25) as server: server.starttls() server.login("", "") server.sendmail("", "", msg.as_string()) def predict_image(img, conf_threshold, iou_threshold): results = model.predict( source=img, conf=conf_threshold, iou=iou_threshold, show_labels=True, show_conf=True, imgsz=640, ) detected_objects = {} for r in results: for obj in r.boxes: category = model.names[int(obj.cls)] detected_objects[category] = detected_objects.get(category, 0) + 1 im_array = r.plot() im = Image.fromarray(im_array[..., ::-1]) send_alert(detected_objects) return im, detected_objects iface = gr.Interface( fn=predict_image, inputs=[ gr.Image(type="pil", label="Upload Image"), gr.Slider(minimum=0, maximum=1, value=0.25, label="Confidence threshold"), gr.Slider(minimum=0, maximum=1, value=0.45, label="IoU threshold"), ], outputs=[ gr.Image(type="pil", label="Result"), gr.JSON(label="Detected Objects") ], ) if __name__ == "__main__": iface.launch(share=True, auth=("123", "123"))At present, the program can detect an image and display the category and number of objects in a dialog box below it. How to improve the program so that it can display the category, confidence level, and location of each target in the dialog box below it (the coordinates of the top left and bottom right corner of the target box).
Glenn JocherUltralytics@yqsl123 to display detailed detection information including confidence and bounding box coordinates in your Gradio interface, modify your
predict_imagefunction to extract this data from the YOLO results. Here's a concise implementation:def predict_image(img, conf_threshold, iou_threshold): results = model.predict( source=img, conf=conf_threshold, iou=iou_threshold, imgsz=640, ) detections = [] for r in results: for obj in r.boxes: detections.append({ "category": model.names[int(obj.cls)], "confidence": round(float(obj.conf), 2), "bbox": [round(x, 2) for x in obj.xyxy[0].tolist()] }) im = Image.fromarray(r.plot()[..., ::-1]) return im, detectionsThis implementation follows the pattern shown in the Gradio integration example while adding detailed detection metadata. For commercial deployments, remember that an Ultralytics Enterprise License is required unless your entire project is open-sourced under AGPL-3.0.
yqsl123
I can't recognize pictures properly after replacing the original program with this program. Could you please give the complete program instead of a certain fragment? By the way, the program includes the ability to send mail, which also displays the category, confidence, and location of the target, which I would like to keep.
Glenn JocherUltralytics(editado)For a complete implementation combining Gradio detection with email alerts, modify your
predict_imagefunction as shown in the Gradio integration example while adding box data extraction:def predict_image(img, conf_threshold, iou_threshold): results = model.predict( source=img, conf=conf_threshold, iou=iou_threshold, imgsz=640, ) detections = [] for r in results: for obj in r.boxes: detections.append({ "category": model.names[int(obj.cls)], "confidence": round(float(obj.conf), 2), "bbox": [round(x, 2) for x in obj.xyxy[0].tolist()] }) im = Image.fromarray(r.plot()[..., ::-1]) if detections: # Only send email when detections exist counts = {} for d in detections: counts[d["category"]] = counts.get(d["category"], 0) + 1 send_alert(counts) # send_alert expects a {category: count} dict return im, detectionsThis maintains your email functionality while properly handling detection outputs. For commercial implementations, remember an Ultralytics Enterprise License is required unless fully open-sourced under AGPL-3.0.
ARAF23
thanks, working
On Fri, Sep 6, 2024 at 10:20 AM Glenn Jocher @.***> wrote:
@ARAF23 https://github.com/ARAF23 it seems like you're encountering an issue with Pydantic in your Gradio setup. Please ensure you're using the latest versions of Gradio and Pydantic. You might also want to set arbitrary_types_allowed=True in your Pydantic model configuration. If the problem persists, try running the example code provided in the Gradio documentation https://docs.ultralytics.com/integrations/gradio/ to verify your setup.
— Reply to this email directly, view it on GitHub https://github.com/orgs/ultralytics/discussions/9131#discussioncomment-10564478, or unsubscribe https://github.com/notifications/unsubscribe-auth/AZ4W6RXE2FW7IPMGHDN5O3TZVEURXAVCNFSM6AAAAABE6N66NKVHI2DSMVQWIX3LMV43URDJONRXK43TNFXW4Q3PNVWWK3TUHMYTANJWGQ2DOOA . You are receiving this because you were mentioned.Message ID: @.*** com>
ARAF23
I was following the video titled " Deployment and Integrations Ultralytics YOLOv8 | Episode 33 " by ultralytics. Everytime when i click on upload or flag, i am having this error :
ERROR: Exception in ASGI application Traceback (most recent call last): File "C:\Users\MSI\AppData\Roaming\Python\Python311\site-packages\pydantic\type_adapter.py", line 279, in _init_core_attrs self._core_schema = _getattr_no_parents(self._type, 'pydantic_core_schema') ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "C:\Users\MSI\AppData\Roaming\Python\Python311\site-packages\pydantic\type_adapter.py", line 121, in _getattr_no_parents raise AttributeError(attribute) AttributeError: pydantic_core_schema
During handling of the above exception, another exception occurred:
Traceback (most recent call last): File "C:\Users\MSI\AppData\Roaming\Python\Python311\site-packages\uvicorn\protocols\http\h11_impl.py", line 406, in run_asgi result = await app( # type: ignore[func-returns-value] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "C:\Users\MSI\AppData\Roaming\Python\Python311\site-packages\uvicorn\middleware\proxy_headers.py", line 70, in call return await self.app(scope, receive, send) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "C:\Users\MSI\AppData\Roaming\Python\Python311\site-packages\fastapi\applications.py", line 1054, in call await super().call(scope, receive, send) File "C:\Users\MSI\AppData\Roaming\Python\Python311\site-packages\starlette\applications.py", line 113, in call await self.middleware_stack(scope, receive, send) File "C:\Users\MSI\AppData\Roaming\Python\Python311\site-packages\starlette\middleware\errors.py", line 187, in call raise exc File "C:\Users\MSI\AppData\Roaming\Python\Python311\site-packages\starlette\middleware\errors.py", line 165, in call await self.app(scope, receive, send) File "C:\Users\MSI\AppData\Roaming\Python\Python311\site-packages\gradio\route_utils.py", line 766, in call
await self.simple_response(scope, receive, send, request_headers=headers) File "C:\Users\MSI\AppData\Roaming\Python\Python311\site-packages\gradio\route_utils.py", line 782, in simple_response await self.app(scope, receive, send) File "C:\Users\MSI\AppData\Roaming\Python\Python311\site-packages\starlette\middleware\exceptions.py", line 62, in call await wrap_app_handling_exceptions(self.app, conn)(scope, receive, send) File "C:\Users\MSI\AppData\Roaming\Python\Python311\site-packages\starlette_exception_handler.py", line 62, in wrapped_app raise exc File "C:\Users\MSI\AppData\Roaming\Python\Python311\site-packages\starlette_exception_handler.py", line 51, in wrapped_app await app(scope, receive, sender) File "C:\Users\MSI\AppData\Roaming\Python\Python311\site-packages\starlette\routing.py", line 715, in call
await self.middleware_stack(scope, receive, send) File "C:\Users\MSI\AppData\Roaming\Python\Python311\site-packages\starlette\routing.py", line 735, in app await route.handle(scope, receive, send) File "C:\Users\MSI\AppData\Roaming\Python\Python311\site-packages\starlette\routing.py", line 288, in handle
await self.app(scope, receive, send) File "C:\Users\MSI\AppData\Roaming\Python\Python311\site-packages\starlette\routing.py", line 76, in app await wrap_app_handling_exceptions(app, request)(scope, receive, send) File "C:\Users\MSI\AppData\Roaming\Python\Python311\site-packages\starlette_exception_handler.py", line 62, in wrapped_app raise exc File "C:\Users\MSI\AppData\Roaming\Python\Python311\site-packages\starlette_exception_handler.py", line 51, in wrapped_app await app(scope, receive, sender) File "C:\Users\MSI\AppData\Roaming\Python\Python311\site-packages\starlette\routing.py", line 73, in app response = await f(request) ^^^^^^^^^^^^^^^^ File "C:\Users\MSI\AppData\Roaming\Python\Python311\site-packages\fastapi\routing.py", line 291, in app solved_result = await solve_dependencies( ^^^^^^^^^^^^^^^^^^^^^^^^^ File "C:\Users\MSI\AppData\Roaming\Python\Python311\site-packages\fastapi\dependencies\utils.py", line 639, in solve_dependencies ) = await request_body_to_args( # body_params checked above ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "C:\Users\MSI\AppData\Roaming\Python\Python311\site-packages\fastapi\dependencies\utils.py", line 810, in request_body_to_args fields_to_extract = get_model_fields(first_field.type) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "C:\Users\MSI\AppData\Roaming\Python\Python311\site-packages\fastapi_compat.py", line 283, in get_model_fields return [ ^ File "C:\Users\MSI\AppData\Roaming\Python\Python311\site-packages\fastapi_compat.py", line 284, in
ModelField(field_info=field_info, name=name) File "", line 6, in init File "C:\Users\MSI\AppData\Roaming\Python\Python311\site-packages\fastapi_compat.py", line 109, in post_init self._type_adapter: TypeAdapter[Any] = TypeAdapter( ^^^^^^^^^^^^ File "C:\Users\MSI\AppData\Roaming\Python\Python311\site-packages\pydantic\type_adapter.py", line 266, in init self._init_core_attrs(rebuild_mocks=False) File "C:\Users\MSI\AppData\Roaming\Python\Python311\site-packages\pydantic\type_adapter.py", line 144, in wrapped return func(self, *args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "C:\Users\MSI\AppData\Roaming\Python\Python311\site-packages\pydantic\type_adapter.py", line 286, in _init_core_attrs self._core_schema = _get_schema(self._type, config_wrapper, parent_depth=self._parent_depth) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "C:\Users\MSI\AppData\Roaming\Python\Python311\site-packages\pydantic\type_adapter.py", line 104, in get_schema schema = gen.generate_schema(type) ^^^^^^^^^^^^^^^^^^^^^^^^^^ File "C:\Users\MSI\AppData\Roaming\Python\Python311\site-packages\pydantic_internal_generate_schema.py", line 655, in generate_schema schema = self._generate_schema_inner(obj) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "C:\Users\MSI\AppData\Roaming\Python\Python311\site-packages\pydantic_internal_generate_schema.py", line 908, in _generate_schema_inner return self._annotated_schema(obj) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "C:\Users\MSI\AppData\Roaming\Python\Python311\site-packages\pydantic_internal_generate_schema.py", line 2025, in _annotated_schema schema = self._apply_annotations(source_type, annotations) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "C:\Users\MSI\AppData\Roaming\Python\Python311\site-packages\pydantic_internal_generate_schema.py", line 2104, in _apply_annotations schema = get_inner_schema(source_type) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "C:\Users\MSI\AppData\Roaming\Python\Python311\site-packages\pydantic_internal_schema_generation_shared.py", line 83, in call schema = self._handler(source_type) ^^^^^^^^^^^^^^^^^^^^^^^^^^ File "C:\Users\MSI\AppData\Roaming\Python\Python311\site-packages\pydantic_internal_generate_schema.py", line 2186, in new_handler schema = metadata_get_schema(source, get_inner_schema) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "C:\Users\MSI\AppData\Roaming\Python\Python311\site-packages\pydantic_internal_generate_schema.py", line 2182, in lambda source, handler: handler(source) ^^^^^^^^^^^^^^^ File "C:\Users\MSI\AppData\Roaming\Python\Python311\site-packages\pydantic_internal_schema_generation_shared.py", line 83, in call schema = self._handler(source_type) ^^^^^^^^^^^^^^^^^^^^^^^^^^ File "C:\Users\MSI\AppData\Roaming\Python\Python311\site-packages\pydantic_internal_generate_schema.py", line 2085, in inner_handler schema = self._generate_schema_inner(obj) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "C:\Users\MSI\AppData\Roaming\Python\Python311\site-packages\pydantic_internal_generate_schema.py", line 929, in _generate_schema_inner return self.match_type(obj) ^^^^^^^^^^^^^^^^^^^^ File "C:\Users\MSI\AppData\Roaming\Python\Python311\site-packages\pydantic_internal_generate_schema.py", line 1029, in match_type return self._match_generic_type(obj, origin) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "C:\Users\MSI\AppData\Roaming\Python\Python311\site-packages\pydantic_internal_generate_schema.py", line 1058, in _match_generic_type return self._union_schema(obj) ^^^^^^^^^^^^^^^^^^^^^^^ File "C:\Users\MSI\AppData\Roaming\Python\Python311\site-packages\pydantic_internal_generate_schema.py", line 1378, in _union_schema choices.append(self.generate_schema(arg)) ^^^^^^^^^^^^^^^^^^^^^^^^^ File "C:\Users\MSI\AppData\Roaming\Python\Python311\site-packages\pydantic_internal_generate_schema.py", line 655, in generate_schema schema = self._generate_schema_inner(obj) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "C:\Users\MSI\AppData\Roaming\Python\Python311\site-packages\pydantic_internal_generate_schema.py", line 929, in _generate_schema_inner return self.match_type(obj) ^^^^^^^^^^^^^^^^^^^^ File "C:\Users\MSI\AppData\Roaming\Python\Python311\site-packages\pydantic_internal_generate_schema.py", line 1038, in match_type return self._unknown_type_schema(obj) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "C:\Users\MSI\AppData\Roaming\Python\Python311\site-packages\pydantic_internal_generate_schema.py", line 558, in _unknown_type_schema raise PydanticSchemaGenerationError( pydantic.errors.PydanticSchemaGenerationError: Unable to generate pydantic-core schema for <class 'starlette.requests.Request'>. Setarbitrary_types_allowed=Truein the model_config to ignore this error or implement__get_pydantic_core_schema__on your type to fully support it.If you got this error by calling handler() within
__get_pydantic_core_schema__then you likely need to callhandler.generate_schema(<some type>)since we do not call__get_pydantic_core_schema__on<some type>otherwise to avoid infinite recursion.For further information visit https://errors.pydantic.dev/2.9/u/schema-for-unknown-type ERROR: Exception in ASGI application Traceback (most recent call last): File "C:\Users\MSI\AppData\Roaming\Python\Python311\site-packages\pydantic\type_adapter.py", line 279, in _init_core_attrs self._core_schema = _getattr_no_parents(self._type, 'pydantic_core_schema') ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "C:\Users\MSI\AppData\Roaming\Python\Python311\site-packages\pydantic\type_adapter.py", line 121, in _getattr_no_parents raise AttributeError(attribute) AttributeError: pydantic_core_schema
During handling of the above exception, another exception occurred:
Traceback (most recent call last): File "C:\Users\MSI\AppData\Roaming\Python\Python311\site-packages\uvicorn\protocols\http\h11_impl.py", line 406, in run_asgi result = await app( # type: ignore[func-returns-value] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "C:\Users\MSI\AppData\Roaming\Python\Python311\site-packages\uvicorn\middleware\proxy_headers.py", line 70, in call return await self.app(scope, receive, send) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "C:\Users\MSI\AppData\Roaming\Python\Python311\site-packages\fastapi\applications.py", line 1054, in call await super().call(scope, receive, send) File "C:\Users\MSI\AppData\Roaming\Python\Python311\site-packages\starlette\applications.py", line 113, in call await self.middleware_stack(scope, receive, send) File "C:\Users\MSI\AppData\Roaming\Python\Python311\site-packages\starlette\middleware\errors.py", line 187, in call raise exc File "C:\Users\MSI\AppData\Roaming\Python\Python311\site-packages\starlette\middleware\errors.py", line 165, in call await self.app(scope, receive, send) File "C:\Users\MSI\AppData\Roaming\Python\Python311\site-packages\gradio\route_utils.py", line 766, in call
await self.simple_response(scope, receive, send, request_headers=headers) File "C:\Users\MSI\AppData\Roaming\Python\Python311\site-packages\gradio\route_utils.py", line 782, in simple_response await self.app(scope, receive, send) File "C:\Users\MSI\AppData\Roaming\Python\Python311\site-packages\starlette\middleware\exceptions.py", line 62, in call await wrap_app_handling_exceptions(self.app, conn)(scope, receive, send) File "C:\Users\MSI\AppData\Roaming\Python\Python311\site-packages\starlette_exception_handler.py", line 62, in wrapped_app raise exc File "C:\Users\MSI\AppData\Roaming\Python\Python311\site-packages\starlette_exception_handler.py", line 51, in wrapped_app await app(scope, receive, sender) File "C:\Users\MSI\AppData\Roaming\Python\Python311\site-packages\starlette\routing.py", line 715, in call
await self.middleware_stack(scope, receive, send) File "C:\Users\MSI\AppData\Roaming\Python\Python311\site-packages\starlette\routing.py", line 735, in app await route.handle(scope, receive, send) File "C:\Users\MSI\AppData\Roaming\Python\Python311\site-packages\starlette\routing.py", line 288, in handle
await self.app(scope, receive, send) File "C:\Users\MSI\AppData\Roaming\Python\Python311\site-packages\starlette\routing.py", line 76, in app await wrap_app_handling_exceptions(app, request)(scope, receive, send) File "C:\Users\MSI\AppData\Roaming\Python\Python311\site-packages\starlette_exception_handler.py", line 62, in wrapped_app raise exc File "C:\Users\MSI\AppData\Roaming\Python\Python311\site-packages\starlette_exception_handler.py", line 51, in wrapped_app await app(scope, receive, sender) File "C:\Users\MSI\AppData\Roaming\Python\Python311\site-packages\starlette\routing.py", line 73, in app response = await f(request) ^^^^^^^^^^^^^^^^ File "C:\Users\MSI\AppData\Roaming\Python\Python311\site-packages\fastapi\routing.py", line 291, in app solved_result = await solve_dependencies( ^^^^^^^^^^^^^^^^^^^^^^^^^ File "C:\Users\MSI\AppData\Roaming\Python\Python311\site-packages\fastapi\dependencies\utils.py", line 639, in solve_dependencies ) = await request_body_to_args( # body_params checked above ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "C:\Users\MSI\AppData\Roaming\Python\Python311\site-packages\fastapi\dependencies\utils.py", line 810, in request_body_to_args fields_to_extract = get_model_fields(first_field.type) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "C:\Users\MSI\AppData\Roaming\Python\Python311\site-packages\fastapi_compat.py", line 283, in get_model_fields return [ ^ File "C:\Users\MSI\AppData\Roaming\Python\Python311\site-packages\fastapi_compat.py", line 284, in
ModelField(field_info=field_info, name=name) File "", line 6, in init File "C:\Users\MSI\AppData\Roaming\Python\Python311\site-packages\fastapi_compat.py", line 109, in post_init self._type_adapter: TypeAdapter[Any] = TypeAdapter( ^^^^^^^^^^^^ File "C:\Users\MSI\AppData\Roaming\Python\Python311\site-packages\pydantic\type_adapter.py", line 266, in init self._init_core_attrs(rebuild_mocks=False) File "C:\Users\MSI\AppData\Roaming\Python\Python311\site-packages\pydantic\type_adapter.py", line 144, in wrapped return func(self, *args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "C:\Users\MSI\AppData\Roaming\Python\Python311\site-packages\pydantic\type_adapter.py", line 286, in _init_core_attrs self._core_schema = _get_schema(self._type, config_wrapper, parent_depth=self._parent_depth) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "C:\Users\MSI\AppData\Roaming\Python\Python311\site-packages\pydantic\type_adapter.py", line 104, in get_schema schema = gen.generate_schema(type) ^^^^^^^^^^^^^^^^^^^^^^^^^^ File "C:\Users\MSI\AppData\Roaming\Python\Python311\site-packages\pydantic_internal_generate_schema.py", line 655, in generate_schema schema = self._generate_schema_inner(obj) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "C:\Users\MSI\AppData\Roaming\Python\Python311\site-packages\pydantic_internal_generate_schema.py", line 908, in _generate_schema_inner return self._annotated_schema(obj) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "C:\Users\MSI\AppData\Roaming\Python\Python311\site-packages\pydantic_internal_generate_schema.py", line 2025, in _annotated_schema schema = self._apply_annotations(source_type, annotations) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "C:\Users\MSI\AppData\Roaming\Python\Python311\site-packages\pydantic_internal_generate_schema.py", line 2104, in _apply_annotations schema = get_inner_schema(source_type) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "C:\Users\MSI\AppData\Roaming\Python\Python311\site-packages\pydantic_internal_schema_generation_shared.py", line 83, in call schema = self._handler(source_type) ^^^^^^^^^^^^^^^^^^^^^^^^^^ File "C:\Users\MSI\AppData\Roaming\Python\Python311\site-packages\pydantic_internal_generate_schema.py", line 2186, in new_handler schema = metadata_get_schema(source, get_inner_schema) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "C:\Users\MSI\AppData\Roaming\Python\Python311\site-packages\pydantic_internal_generate_schema.py", line 2182, in lambda source, handler: handler(source) ^^^^^^^^^^^^^^^ File "C:\Users\MSI\AppData\Roaming\Python\Python311\site-packages\pydantic_internal_schema_generation_shared.py", line 83, in call schema = self._handler(source_type) ^^^^^^^^^^^^^^^^^^^^^^^^^^ File "C:\Users\MSI\AppData\Roaming\Python\Python311\site-packages\pydantic_internal_generate_schema.py", line 2085, in inner_handler schema = self._generate_schema_inner(obj) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "C:\Users\MSI\AppData\Roaming\Python\Python311\site-packages\pydantic_internal_generate_schema.py", line 929, in _generate_schema_inner return self.match_type(obj) ^^^^^^^^^^^^^^^^^^^^ File "C:\Users\MSI\AppData\Roaming\Python\Python311\site-packages\pydantic_internal_generate_schema.py", line 1029, in match_type return self._match_generic_type(obj, origin) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "C:\Users\MSI\AppData\Roaming\Python\Python311\site-packages\pydantic_internal_generate_schema.py", line 1058, in _match_generic_type return self._union_schema(obj) ^^^^^^^^^^^^^^^^^^^^^^^ File "C:\Users\MSI\AppData\Roaming\Python\Python311\site-packages\pydantic_internal_generate_schema.py", line 1378, in _union_schema choices.append(self.generate_schema(arg)) ^^^^^^^^^^^^^^^^^^^^^^^^^ File "C:\Users\MSI\AppData\Roaming\Python\Python311\site-packages\pydantic_internal_generate_schema.py", line 655, in generate_schema schema = self._generate_schema_inner(obj) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "C:\Users\MSI\AppData\Roaming\Python\Python311\site-packages\pydantic_internal_generate_schema.py", line 929, in _generate_schema_inner return self.match_type(obj) ^^^^^^^^^^^^^^^^^^^^ File "C:\Users\MSI\AppData\Roaming\Python\Python311\site-packages\pydantic_internal_generate_schema.py", line 1038, in match_type return self._unknown_type_schema(obj) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "C:\Users\MSI\AppData\Roaming\Python\Python311\site-packages\pydantic_internal_generate_schema.py", line 558, in _unknown_type_schema raise PydanticSchemaGenerationError( pydantic.errors.PydanticSchemaGenerationError: Unable to generate pydantic-core schema for <class 'starlette.requests.Request'>. Setarbitrary_types_allowed=Truein the model_config to ignore this error or implement__get_pydantic_core_schema__on your type to fully support it.If you got this error by calling handler() within
__get_pydantic_core_schema__then you likely need to callhandler.generate_schema(<some type>)since we do not call__get_pydantic_core_schema__on<some type>otherwise to avoid infinite recursion.For further information visit https://errors.pydantic.dev/2.9/u/schema-for-unknown-type
Glenn JocherUltralytics@ARAF23 it seems like you're encountering an issue with Pydantic in your Gradio setup. Please ensure you're using the latest versions of Gradio and Pydantic. You might also want to set
arbitrary_types_allowed=Truein your Pydantic model configuration. If the problem persists, try running the example code provided in the Gradio documentation to verify your setup.
SoulPerforms
Hi, i'm new to this, I've finished training my yolov8 model and want to deploy it on Hugging Face Space with gradio. How to do video inference using gradio, so that i can input the video and make the model process it and display the output. Thanks
Glenn JocherUltralytics@SoulPerforms hey there! 😊 Welcome to the exciting world of deploying models with Gradio on Hugging Face Spaces! For video inference using YOLOv8 with Gradio, you'll need to slightly modify the approach since video processing differs from image processing. Currently, Gradio doesn't natively support video streams as input. A common workaround is to process the video frame by frame and display the output frames as a sequence or compile them back into a video.
Here's a basic idea of how you might approach this:
- Extract Frames from Video: First, you'll need to decompose your video into frames.
- Process Each Frame: Use YOLOv8 to process each extracted frame.
- Display or Compile Frames: Show the processed frames in Gradio or compile them back into a video format.
import cv2 from PIL import Image import gradio as gr from ultralytics import YOLO # Load your YOLOv8 model model = YOLO("yolov8n.pt") def process_video(video_path): cap = cv2.VideoCapture(video_path) processed_frames = [] while cap.isOpened(): ret, frame = cap.read() if not ret: break # Model expects PIL Image format pil_img = Image.fromarray(frame[..., ::-1]) # Convert BGR to RGB result = model.predict(source=pil_img) for r in result: im_array = r.plot() processed_frames.append(Image.fromarray(im_array[..., ::-1])) # Convert RGB back to BGR cap.release() # You may choose to display each frame or compile them back using cv2 or a similar library return processed_frames[-1] # Example, returning the last processed frame # Define your Gradio interface here # Due to Gradio's limitations on video processing, you might need to adjust how you feed and display your video in the interface # For detailed implementation, refer to the docs and adapt the example code for images to work with video frames.This example provides a starting point for processing video frames. Adjustments will be necessary to tailor the process to your specific requirements, like handling video uploads and generating a final video output in Gradio.
For detailed Gradio interfaces and displaying results, refer to Gradio's documentation and keep an eye on updates for native video support! 🚀
SoulPerforms
thank you for your reply, based on your code, i have successfully deployed my trained yolov8 model to hugging face space to do video inference by generating frame one by one.
SoulPerforms
here is a link to my model: https://huggingface.co/spaces/SoulPerforms/Fire_Detection_YOLOv8_Model_Inference_with_Gradio
Glenn JocherUltralyticsHey there! 😊
Fantastic job deploying your YOLOv8 model with Gradio on Hugging Face Spaces! It's great to see your model in action, making video inference accessible and interactive for everyone.
For anyone looking to take advantage of this awesome Gradio integration for quick and user-friendly model demonstrations or experiments, check out the live example at Fire Detection YOLOv8 Model Inference with Gradio. Adjust detection parameters in real-time and see your model's performance on various inputs! 🔥🤖
Happy detecting!
Rifeet
Hello @SoulPerforms, if its okay for you can please share the interface code
SoulPerforms
you can go to files tab on my huggingface spaces and check the code
Rifeet
Thank you so much @SoulPerforms, you saved my life
lishihong-1
Why does the video in the detection interface play slowly? What should I do to use the video stream detection in gradio?
Glenn JocherUltralyticsHey there! 👋 If your video playback in the Gradio interface is slower than expected, this might be due to the processing time required for each frame. Here are a couple of tips to help speed things up:
-
Optimize Model Inference: Reduce the image size for detection (
imgsz), or you can try using a model optimized for speed likeyolov8n.pt. -
Adjust Gradio Code: If you're processing video streams, consider adding some buffering or optimizations in the frame handling to improve performance.
For real-time video stream detection, using Gradio can be a challenge since it doesn't naturally support streaming video feeds. I recommend employing a frame by frame approach where each frame is processed individually or integrating with another tool that supports video streaming.
# Simplified example assuming frame extraction is handled elsewhere from PIL import Image import gradio as gr from ultralytics import YOLO model = YOLO("yolov8n.pt") # Load speed-optimized model def process_frame(image): result = model.predict(image) processed_image = Image.fromarray(result[0].plot()) return processed_image iface = gr.Interface(fn=process_frame, inputs="image", outputs="image") iface.launch()This setup assumes you have a mechanism to feed video frames one at a time. Hope this helps! Let me know if you have any more questions. 😊
-