Food recognition
Image classification on high-performance MCU.
Food recognition can be used in a wide range of applications, such as home appliances (smart fridges, microwave ovens), restaurants, hospitals, or in the food industry. Based on a FD-MobileNet model, the application can recognize 18 different types of food and beverages including pizza, beer, and fries, among many others.
Approach
- We used of a camera module (B-CAMS-OMV) to capture the scene
- We selected a pre-trained FD-Mobilenet NN model to perform food recognition
- This model is already integrated in the function pack FP-AI-VISION1 (made for STM32H747 discovery kit)
- The model was then optimized using STM32Cube.AI
- We selected a pre-trained FD-Mobilenet NN model to perform food recognition
- This model is already integrated in the function pack FP-AI-VISION1 (made for STM32H747 discovery kit)
- The model was then optimized using STM32Cube.AI
Sensor
Vision: camera module bundle (reference: B-CAMS-OMV)
Data
Data format
- 18 classes: "Apple Pie", "Beer", "Caesar Salad", "Cappuccino", "Cheesecake", "Chicken Wings", "Chocolate Cake", "Coke", "Cup Cakes", "Donuts", "French Fries", "Hamburger", "Hot Dog", "Lasagna", "Pizza", "Risotto", "Spaghetti Bolognese", "Steak"
- RGB color image
- 18 classes: "Apple Pie", "Beer", "Caesar Salad", "Cappuccino", "Cheesecake", "Chicken Wings", "Chocolate Cake", "Coke", "Cup Cakes", "Donuts", "French Fries", "Hamburger", "Hot Dog", "Lasagna", "Pizza", "Risotto", "Spaghetti Bolognese", "Steak"
- RGB color image
Results
We provide two different networks, which offer a specific trade-off between inference time and accuracy.
Model: Standard Convolutional Neural Network quantized
Input size: 224x224x3
Memory footprint:
132 KB Flash for weights
148 KBRAM for activations
Accuracy: 72.8%
Performance on STM32H747 (High-Perf) @ 400 MHz
Inference time: 79 ms
Frame rate: 11.8 fps
Model: Optimized Convolutional Neural Network quantized
Input size: 224x224x3
Memory footprint:
148 KB Flash for weights
199 KBRAM for activations
Accuracy: 77,5%
Performance on STM32H747 (High-Perf) @ 400 MHz
Inference time: 145 ms
Frame rate: 6.6 fps
On-board validation summary of information for a food recognition example
Model: Standard Convolutional Neural Network quantized
Input size: 224x224x3
Memory footprint:
132 KB Flash for weights
148 KBRAM for activations
Accuracy: 72.8%
Performance on STM32H747 (High-Perf) @ 400 MHz
Inference time: 79 ms
Frame rate: 11.8 fps
Model: Optimized Convolutional Neural Network quantized
Input size: 224x224x3
Memory footprint:
148 KB Flash for weights
199 KBRAM for activations
Accuracy: 77,5%
Performance on STM32H747 (High-Perf) @ 400 MHz
Inference time: 145 ms
Frame rate: 6.6 fps
Resources
Optimized with STM32Cube.AI
A free STM32Cube expansion package, X-CUBE-AI allows developers to convert pretrained AI algorithms automatically, such as neural network and machine learning models, into optimized C code for STM32.
Compatible with STM32H7 series
The STM32 family of 32-bit microcontrollers based on the Arm Cortex®-M processor is designed to offer new degrees of freedom to MCU users. It offers products combining very high performance, real-time capabilities, digital signal processing, low-power / low-voltage operation, and connectivity, while maintaining full integration and ease of development.
