Are you sure you want to delete this access key?
This guide demonstrates how to perform inference using an Ultralytics YOLOv8 model exported to the TensorFlow Lite (TFLite) format. TFLite is a popular choice for deploying machine learning models on mobile, embedded, and IoT devices due to its optimization for on-device inference with low latency and a small binary size. This example supports FP32, FP16, and INT8 quantized TFLite models.
Before running inference, you need to install the necessary TFLite interpreter package. Choose the appropriate package based on your hardware (CPU or GPU).
tflite-runtime
(Recommended for Edge Devices)The tflite-runtime
package is a smaller package that includes the bare minimum required to run inferences with TensorFlow Lite, primarily the Interpreter
Python class. It's ideal for resource-constrained environments like Raspberry Pi or Coral Edge TPU.
pip install tflite-runtime
For more details, refer to the official TFLite Python quickstart guide.
tensorflow
Package (CPU or GPU)Alternatively, you can install the full TensorFlow package. This includes the TFLite interpreter along with the complete TensorFlow library.
CPU-Only: Suitable if you don't have an NVIDIA GPU or don't need GPU acceleration.
pip install tensorflow
GPU Support: To leverage NVIDIA GPU acceleration for potentially faster inference, install tensorflow
with GPU support. Ensure you have the necessary NVIDIA drivers and CUDA toolkit installed.
# Check TensorFlow documentation for specific CUDA/cuDNN version requirements
pip install tensorflow[and-cuda] # Or follow specific instructions on TF website
Visit the official TensorFlow installation guide for detailed instructions, including GPU setup.
Follow these steps to run inference with your exported YOLOv8 TFLite model.
Export YOLOv8 Model to TFLite:
First, export your trained Ultralytics YOLOv8 model (e.g., yolov8n.pt
) to the TFLite format using the yolo export
command. This example exports an INT8 quantized model for optimal performance on edge devices. You can also export FP32 or FP16 models by adjusting the format
and quantization arguments. Refer to the Ultralytics Export mode documentation for more options.
yolo export model=yolov8n.pt imgsz=640 format=tflite int8=True # Exports yolov8n_saved_model/yolov8n_full_integer_quant.tflite
The export process will create a directory (e.g., yolov8n_saved_model
) containing the .tflite
model file and potentially a metadata.yaml
file with class names and other model details.
Run Inference Script:
Execute the provided Python script (main.py
) to perform inference on an image. Adjust the arguments as needed for your specific model path, image source, confidence threshold, and IoU threshold.
python main.py \
--model yolov8n_saved_model/yolov8n_full_integer_quant.tflite \
--img image.jpg \
--conf 0.25 \
--iou 0.45 \
--metadata yolov8n_saved_model/metadata.yaml
--model
: Path to the exported .tflite
model file.--img
: Path to the input image for detection.--conf
: Minimum confidence threshold for detections (e.g., 0.25).--iou
: Intersection over Union (IoU) threshold for Non-Maximum Suppression (NMS).--metadata
: Path to the metadata.yaml
file generated during export (contains class names).The script will process the input image using the specified TFLite model and display the image with bounding boxes drawn around detected objects. Each box will be labeled with the predicted class name and confidence score.
This example provides a straightforward way to deploy Ultralytics YOLOv8 models on devices supporting TFLite, enabling efficient object detection in various applications. Explore different quantization options and model sizes to find the best balance between performance and accuracy for your specific use case.
Contributions to enhance this example or add new functionalities are welcome! Feel free to fork the Ultralytics repository, make your changes, and submit a pull request.
Press p or to see the previous file or, n or to see the next file
Browsing data directories saved to S3 is possible with DAGsHub. Let's configure your repository to easily display your data in the context of any commit!
ultralytics is now integrated with AWS S3!
Are you sure you want to delete this access key?
Browsing data directories saved to Google Cloud Storage is possible with DAGsHub. Let's configure your repository to easily display your data in the context of any commit!
ultralytics is now integrated with Google Cloud Storage!
Are you sure you want to delete this access key?
Browsing data directories saved to Azure Cloud Storage is possible with DAGsHub. Let's configure your repository to easily display your data in the context of any commit!
ultralytics is now integrated with Azure Cloud Storage!
Are you sure you want to delete this access key?
Browsing data directories saved to S3 compatible storage is possible with DAGsHub. Let's configure your repository to easily display your data in the context of any commit!
ultralytics is now integrated with your S3 compatible storage!
Are you sure you want to delete this access key?