Compiling & Quantizing Ultralytics YOLOv5 For Better Performance With Deci

At Ultralytics we commercially partner with other startups to help us fund the research and development of our open-source tools, like YOLOv5, to keep them free for everybody. This article may contain affiliate links to those partners.

Update: NVIDIA acquired Deci in 2024, and deci.ai now redirects to NVIDIA. The Deci platform steps below are kept for reference.

The Deci platform includes free tools for easily managing, optimizing, and deploying your YOLOv5 models in any production environment. Deci supports all popular deep learning frameworks, such as TensorFlow, PyTorch, Keras, and ONNX. All you need is our web-based platform or our Python client to run it from your code.

Why Deci?#

You can use Deci not only for exporting but also for pruning and quantization of the model.

Deci provides a convenient interface for exporting in any format and comparing performance between the original and converted models. Users can choose to further optimize their models by quantization.

With Deci You Can:#

Improve Inference Performance By Up To 10x#

Automatically compile and quantize your models and evaluate different production settings to achieve better latency, throughput, and reduction of the model size and memory footprint on your hardware.

Find The Best Inference Hardware For Your Application#

Benchmark your model's performance on various hardware (including edge) devices with a button. Eliminate the need to manually set up and test multiple hardware and production settings.

Deploy With A Few Lines Of Code#

Leverage Deci's Python-based inference engine. Compatible with multiple frameworks and hardware types.

First-Time Setup#

Step 1#

Open your free account.

Step 2#

To start optimizing your pre-trained YOLOv5 model, you will need to convert it to ONNX format. See the YOLOv5 export tutorial for instructions on how to convert your model to ONNX format.

Step 3#

Go to the "Lab" tab and click the "New Model" button in the top right part of the screen to upload your YOLOv5 ONNX model.

Follow the steps of the model upload wizard to select your target hardware as well as desired batch size and quantization level for the model compilation.

After filling in the relevant information, click "Start". The Deci platform will automatically perform a runtime optimization of your Ultralytics YOLOv5 model for the hardware you selected as well as benchmark your model on various hardware types. This process takes approximately 10 minutes.

Once done, a new row will appear on your screen underneath the baseline model you previously uploaded. Here you can see the optimized version of your pre-trained Ultralytics YOLOv5 model.

What's Next?#

You can then download your optimized model by clicking on the "Deploy" button.

You will then be prompted to download your model and receive instructions on how to install and use Infery - Deci's runtime inference engine.

The use of Infery is optional. You can get the raw Python files and use them with any other inference engine of your choice.

Explore the optimization and benchmark results on the "Insights" tab.

Ready To Get Started?#

Before wrapping up, let’s discuss some of the advantages Deci offers:

• Optimize your model’s inference throughput and latency without compromising accuracy
• Allows you to optimize models from all the popular frameworks
• Supports models targeted at any deep learning task
• Supports deployment on popular CPU and GPU machines
• Benchmarks the fitness of your model on different hardware hosts and cloud providers
• Gets uploaded models ready for serving, inference, and deployment

As you have just seen, you can double the performance of an Ultralytics YOLOv5 model in about 15 minutes. The Deci platform is easy and intuitive to use.

Any questions? Join our community and leave your question today!