The handwriting recognition demo can identify characters, including letters and numbers. The typical use case for this demo is a smartwatch that allows users to compose specific commands (for ex. chat, call, mail, etc.) or to type text messages quickly. It can also be used as a way to interact with smart appliances.

Approach

The handwriting recognition demo recognizes letters and numbers written on the small touch screen (smartwatch) available on the STM32L562E MCU discovery kit.
  • The touch screen is captured as an image to be classified by the neural network (NN)
  • Each character or letter is recognized as a composition of specific commands
  • The demo runs on the STM32L562E discovery kit with an NN inference time for each character

The model can be re-trained thanks to the STM32 Model Zoo.

Sensor

The demo uses the touch screen as an input for the user.
The model zoo allows users to test the camera module bundle (reference: B-CAMS-OMV).

Data

Dataset
In STM32 model zoo, the model is trained on a subset version of the  EMNIST dataset. 
In this experiment, only the ten-digit classes [0;9] and the capital letters of the alphabet [A-Z] were kept from the MatLab version of the EMNIST ByClass dataset.
For the demo, the dataset was enriched with images captured from the touch screen on the ST board.

Data format
The dataset is made of:
  • uppercase letters from A to Z
  • digits from 0 to 9

The dataset contains 28 x 28 pixels of grayscale images organized in 36 balanced classes.

Results

Model ST MNIST
Input size: 28x28x1
Memory footprint:
Float model:
38 Kbytes of flash memory for weights
30 Kbytes ofRAM for activations
Quantized model:
10 Kbytes of flash memory for weights
14 Kbytes ofRAM for activations
Accuracy:
Float model: 93.48%
Quantized model: 93.39%
Performance on STM32L562E @ 110 MHz 
Float model:
Inference time: 83 ms
Frame rate: 12 fps
Quantized model:
Inference time: 29 ms
Frame rate: 34 fps

use-case-stm32cube-ai-handwriting-confusion-matrix use-case-stm32cube-ai-handwriting-confusion-matrix use-case-stm32cube-ai-handwriting-confusion-matrix

Confusion matrix

Model repository
STM32 MODEL ZOO
STM32 MODEL ZOO
Optimized with
STM32Cube.AI
STM32Cube.AI
Compatible with
STM32L4, L5, U5, H7 series
STM32L4, L5, U5, H7 series

Resources

Model repository STM32 MODEL ZOO

A collection of reference AI models optimized to run on ST devices with associated deployment scripts. The model zoo is a valuable resource to add edge AI capabilities to embedded applications.

STM32 MODEL ZOO STM32 MODEL ZOO STM32 MODEL ZOO

Optimized with STM32Cube.AI

A free STM32Cube expansion package, X-CUBE-AI allows developers to convert pretrained AI algorithms automatically, such as neural network and machine learning models, into optimized C code for STM32.

STM32Cube.AI STM32Cube.AI STM32Cube.AI

Compatible with STM32L4, L5, U5, H7 series

The STM32 family of 32-bit microcontrollers based on the Arm Cortex®-M processor is designed to offer new degrees of freedom to MCU users. It offers products combining very high performance, real-time capabilities, digital signal processing, low-power / low-voltage operation, and connectivity, while maintaining full integration and ease of development.

STM32L4, L5, U5, H7 series STM32L4, L5, U5, H7 series STM32L4, L5, U5, H7 series

You might also be interested by

Entertainment | Image recognition | Vision | STM32Cube.AI | Demo | Tutorial | GitHub | Video

Smart mirrors for fitness: pose estimation and multi-person tracking

Track and analyze users' body movements to provide feedback on exercise with STM32N6 at 28 FPS.

Object detection | Vision | STM32Cube.AI | Idea | GitHub | Video | Dataset | Smart building

Personal protective equipment detection

Detection of personal protective equipment on workers using an object detection AI model.

Smart city | Smart home | Smart office | Context awareness | Vision | AI for Linux | Idea | Video

People detection and counting solution

Optimized computer vision using an MPU running at 8 FPS.