Food recognition can be used in a wide range of applications, such as home appliances (smart fridges, microwave ovens), restaurants, hospitals, or in the food industry. Based on a FD-MobileNet model, the application can recognize 18 different types of food and beverages including pizza, beer, and fries, among many others.

Approach

- We used of a camera module (B-CAMS-OMV) to capture the scene 
- We selected a pre-trained FD-Mobilenet NN model to perform food recognition
- This model is already integrated in the function pack FP-AI-VISION1 (made for STM32H747 discovery kit)
- The model was then optimized using STM32Cube.AI

Sensor

Vision: camera module bundle (reference: B-CAMS-OMV)

Data

Data format
- 18 classes: "Apple Pie", "Beer", "Caesar Salad", "Cappuccino", "Cheesecake", "Chicken Wings", "Chocolate Cake", "Coke", "Cup Cakes", "Donuts", "French Fries", "Hamburger", "Hot Dog", "Lasagna", "Pizza", "Risotto", "Spaghetti Bolognese", "Steak" 
- RGB color image 

Results

We provide two different networks, which offer a specific trade-off between inference time and accuracy. 
Model: Standard Convolutional Neural Network quantized
Input size: 224x224x3
Memory footprint:
132 KB Flash for weights
148 KBRAM for activations
Accuracy: 72.8%
Performance on STM32H747 (High-Perf) @ 400 MHz
Inference time: 79 ms
Frame rate:  11.8 fps
Model: Optimized Convolutional Neural Network quantized
Input size: 224x224x3
Memory footprint:
148 KB Flash for weights
199 KBRAM for activations
Accuracy: 77,5%
Performance on STM32H747 (High-Perf) @ 400 MHz
Inference time: 145 ms
Frame rate:  6.6 fps

RESULTS-confusion-matrix-WIP RESULTS-confusion-matrix-WIP RESULTS-confusion-matrix-WIP

On-board validation summary of information for a food recognition example

Resources

Optimized with STM32Cube.AI

A free STM32Cube expansion package, X-CUBE-AI allows developers to convert pretrained AI algorithms automatically, such as neural network and machine learning models, into optimized C code for STM32.

STM32Cube.AI STM32Cube.AI STM32Cube.AI

Compatible with STM32H7 series

The STM32 family of 32-bit microcontrollers based on the Arm Cortex®-M processor is designed to offer new degrees of freedom to MCU users. It offers products combining very high performance, real-time capabilities, digital signal processing, low-power / low-voltage operation, and connectivity, while maintaining full integration and ease of development.

STM32H7 series STM32H7 series STM32H7 series

You might also be interested by

Entertainment | Image recognition | Vision | STM32Cube.AI | Demo | Tutorial | GitHub | Video

Smart mirrors for fitness: pose estimation and multi-person tracking

Track and analyze users' body movements to provide feedback on exercise with STM32N6 at 28 FPS.

Object detection | Vision | STM32Cube.AI | Idea | GitHub | Video | Dataset | Smart building

Personal protective equipment detection

Detection of personal protective equipment on workers using an object detection AI model.

Smart city | Smart home | Smart office | Context awareness | Vision | AI for Linux | Idea | Video

People detection and counting solution

Optimized computer vision using an MPU running at 8 FPS.