Export and optimize Ultralytics YOLOv8 for inference on Intel OpenVINO

In this blog post, we'll be taking a look at how you can export and optimize your pre-trained or custom-trained Ultralytics YOLOv8 model for inference using OpenVINO. If you're using an Intel-based system, whether it’s a CPU or GPU, this guide will show you how to significantly speed up your model with minimal effort.

Why optimize Ultralytics YOLOv8 with OpenVINO?#

Optimizing your Ultralytics YOLOv8 model with OpenVINO can provide up to a 3x speed increase on inference tasks, particularly if you're running an Intel CPU. This performance boost can make a huge difference in real-time applications, from object detection to segmentation and security systems.

Steps to export and optimize your Ultralytics YOLOv8 model#

Understanding the process#

First things first, let’s break down the process. We're going to convert a PyTorch model to the OpenVINO format and optimize it. At the time of the video, this conversion went through ONNX as an intermediate step; current Ultralytics versions convert directly from PyTorch. This process involves a few straightforward steps and can be applied to various models and formats including TensorFlow, PyTorch, and ONNX.

Exporting the model#

Jumping into the Ultralytics documentation, we find that exporting a YOLOv8 model involves using the export method from the Ultralytics framework. This method converts our model from PyTorch and optimizes it for OpenVINO. The result is a model that runs significantly faster on Intel hardware.

Installing dependencies#

Before running the export script, you’ll need to ensure that all necessary dependencies are installed. These include the Ultralytics library and OpenVINO. Installing these packages is a simple process that can be done via pip, the Python package installer.

Running the export script#

Once your environment is set up, you can run your export script. This script will convert your PyTorch model to OpenVINO. The process is straightforward and involves calling a single function to handle the export. The Ultralytics framework makes it easy to convert and optimize your models, ensuring you get the best performance with minimal hassle.

Fig 1. Nicolai Nielsen outlining how to run the export script.

Comparing performance#

After exporting, it’s essential to compare the performance of the original and optimized models. By benchmarking the inference time of both models, you can clearly see the performance gains. Typically, the OpenVINO model will show a significant reduction in inference time compared to the original PyTorch model. This is especially true for larger models where the performance boost is most noticeable.

Real-world application and benefits#

Optimizing Ultralytics YOLOv8 models with OpenVINO is particularly beneficial for applications requiring real-time processing. Here are a few examples:

• Security systems: Real-time object detection can alert security personnel instantly, enhancing safety and responsiveness.
• Automated vehicles: Faster inference speeds improve the responsiveness of autonomous driving systems, making them safer and more reliable.
• Healthcare: Quick image processing for diagnostic tools can save lives by providing faster results, allowing for timely interventions.

By implementing these optimizations, you not only improve performance but also enhance the reliability and efficiency of your applications. This can lead to better user experiences, increased productivity, and more innovative solutions.

Wrapping up#

Exporting and optimizing an Ultralytics YOLOv8 model for OpenVINO is an effective way to use Intel hardware for faster and more efficient AI applications. With just a few simple steps, you can significantly improve your model’s performance and apply it to real-world scenarios effectively.

Make sure to check out more tutorials and guides from Ultralytics, such as the OpenVINO on CPU lesson in Ultralytics Academy, to keep enhancing your AI projects. Visit our GitHub repository and join the Ultralytics community for more insights and updates.

Remember, optimizing your models is not just about speed—it's about unlocking new possibilities and ensuring your AI solutions are robust, efficient, and ready for the future.