ultralytics/examples/YOLOv8-MNN-CPP/README.md

YOLOv8 MNN Inference in C++

Welcome to the Ultralytics YOLOv8 MNN Inference example in C++! This guide will help you get started with leveraging the powerful Ultralytics YOLOv8 models using the Alibaba MNN inference engine in your C++ projects. Whether you're looking to enhance performance on CPU hardware or add flexibility to your applications, this example provides a solid foundation. Learn more about optimizing models and deployment strategies on the Ultralytics blog.

🌟 Features

• 🚀 Model Format Support: Native support for the MNN format.
• ⚡ Precision Options: Run models in FP32, FP16 (half-precision), and INT8 (model quantization) precisions for optimized performance and reduced resource consumption.
• 🔄 Dynamic Shape Loading: Easily handle models with dynamic input shapes, a common requirement in many computer vision tasks.
• 📦 Flexible API Usage: Choose between MNN's high-level Express API for a user-friendly interface or the lower-level Interpreter API for fine-grained control.

📋 Dependencies

To ensure smooth execution, please make sure you have the following dependencies installed:

Dependency	Version	Description
MNN	>=2.0.0	The core inference engine from Alibaba.
C++	>=14	A modern C++ compiler supporting C++14 features.
CMake	>=3.12.0	Cross-platform build system generator required for building MNN and the example.
OpenCV	Optional	Used for image loading and preprocessing within the example (built with MNN).

⚙️ Build Instructions

Follow these steps to build the project:

1. Clone the Ultralytics repository:

   git clone https://github.com/ultralytics/ultralytics.git
   cd ultralytics/examples/YOLOv8-MNN-CPP
   

2. Clone the Alibaba MNN repository:

   git clone https://github.com/alibaba/MNN.git
   cd MNN
   

3. Build the MNN library:

   # Create build directory
   mkdir build && cd build
   
   # Configure CMake (enable OpenCV integration, disable shared libs, enable image codecs)
   cmake -DMNN_BUILD_OPENCV=ON -DBUILD_SHARED_LIBS=OFF -DMNN_IMGCODECS=ON ..
   
   # Build the library (use -j flag for parallel compilation)
   make -j$(nproc) # Use nproc for Linux, sysctl -n hw.ncpu for macOS
   

   Note: If you encounter issues during the build process, consult the official MNN documentation for detailed build instructions and troubleshooting tips.

4. Copy the required MNN libraries and headers to the example project directory:

   # Navigate back to the example directory
   cd ../..
   
   # Create directories for libraries and headers if they don't exist
   mkdir -p libs include
   
   # Copy static libraries
   cp MNN/build/libMNN.a libs/                 # Main MNN library
   cp MNN/build/express/libMNN_Express.a libs/ # MNN Express API library
   cp MNN/build/tools/cv/libMNNOpenCV.a libs/  # MNN OpenCV wrapper library
   
   # Copy header files
   cp -r MNN/include .
   cp -r MNN/tools/cv/include . # MNN OpenCV wrapper headers
   

   Note:

   • The library file extensions (.a for static) and paths might vary based on your operating system (e.g., use .lib on Windows) and build configuration. Adjust the commands accordingly.
   • This example uses static linking (.a files). If you built shared libraries (.so, .dylib, .dll), ensure they are correctly placed or accessible in your system's library path.

5. Create a build directory for the example project and compile using CMake:

   mkdir build && cd build
   cmake ..
   make
   

🔄 Exporting YOLOv8 Models

To use your Ultralytics YOLOv8 model with this C++ example, you first need to export it to the MNN format. This can be done easily using the yolo export command provided by the Ultralytics Python package.

Refer to the Ultralytics Export documentation for detailed instructions and options.

# Export a YOLOv8n model to MNN format with input size 640x640
yolo export model=yolov8n.pt imgsz=640 format=mnn

Alternatively, you can use the MNNConvert tool provided by MNN:

# Assuming MNNConvert is built and in your PATH or MNN build directory
# Convert an ONNX model (first export YOLOv8 to ONNX)
yolo export model=yolov8n.pt format=onnx
./MNN/build/MNNConvert -f ONNX --modelFile yolov8n.onnx --MNNModel yolov8n.mnn --bizCode biz

For more details on model conversion using MNN tools, see the MNN Convert documentation.

🛠️ Usage

Ultralytics CLI in Python (for comparison)

You can verify the exported MNN model using the Ultralytics Python package for a quick check.

Download an example image:

wget https://ultralytics.com/images/bus.jpg

Run prediction using the MNN model:

yolo predict model=yolov8n.mnn source=bus.jpg

Expected Python Output:

ultralytics/examples/YOLOv8-MNN-CPP/assets/bus.jpg: 640x640 4 persons, 1 bus, 84.6ms
Speed: 9.7ms preprocess, 128.7ms inference, 12.4ms postprocess per image at shape (1, 3, 640, 640)
Results saved to runs/detect/predict

(Note: Speed and specific detections might vary based on hardware and model version)

MNN Express API in C++

This example uses the higher-level Express API for simpler inference code.

./build/main yolov8n.mnn bus.jpg

Expected C++ Express API Output:

The device supports: i8sdot:0, fp16:0, i8mm: 0, sve2: 0, sme2: 0
Detection: box = {48.63, 399.30, 243.65, 902.90}, class = person, score = 0.86
Detection: box = {22.14, 228.36, 796.07, 749.74}, class = bus, score = 0.86
Detection: box = {669.92, 375.82, 809.86, 874.41}, class = person, score = 0.86
Detection: box = {216.01, 405.24, 346.36, 858.19}, class = person, score = 0.82
Detection: box = {-0.11, 549.41, 62.05, 874.88}, class = person, score = 0.33
Result image write to `mnn_yolov8_cpp.jpg`.
Speed: 35.6ms preprocess, 386.0ms inference, 68.3ms postprocess

(Note: Speed and specific detections might vary based on hardware and MNN configuration)

MNN Interpreter API in C++

This example uses the lower-level Interpreter API, offering more control over the inference process.

./build/main_interpreter yolov8n.mnn bus.jpg

Expected C++ Interpreter API Output:

The device supports: i8sdot:0, fp16:0, i8mm: 0, sve2: 0, sme2: 0
Detection: box = {48.63, 399.30, 243.65, 902.90}, class = person, score = 0.86
Detection: box = {22.14, 228.36, 796.07, 749.74}, class = bus, score = 0.86
Detection: box = {669.92, 375.82, 809.86, 874.41}, class = person, score = 0.86
Detection: box = {216.01, 405.24, 346.36, 858.19}, class = person, score = 0.82
Result image written to `mnn_yolov8_cpp.jpg`.
Speed: 26.0ms preprocess, 190.9ms inference, 58.9ms postprocess

(Note: Speed and specific detections might vary based on hardware and MNN configuration)

❤️ Contributions

We hope this example helps you integrate Ultralytics YOLOv8 with MNN into your C++ projects effortlessly! Contributions to improve this example or add new features are highly welcome. Please see the Ultralytics contribution guidelines for more information on how to get involved.

For further guides, tutorials, and documentation on Ultralytics YOLO models and tools, visit the main Ultralytics documentation. Happy coding! 🚀