STM32 – Neural Networks, AI, Machine Learning & Predictive maintenance

If you like it, share it

Local Predictive Maintenance is one of the topic of this early 2020.
Local Predictive Maintenance applications are essential to avoid clogging Internet communication lines.
A huge quantity of data would be send to a cloud (mems/sensors data) who should process them and then give us a diagnosis, this mode is not practicable, it would quickly clog Internet communications.
For this simple reason (avoid clogging in Internet lines), today we tend to process sensor/mems data locally to have a system capable of helping the end user in the predictive diagnosis of faults that may arise.
Last but not least, today is possible create a local predictive maintenance, thanks to the computing power of STM32 & STM32MP1.

We want to be very clear, many people present this application as simple, is not true, because is necessary several steps and therefore time and resources to devote, before arriving at a real application.


Explaining Neural Networks (NN) and Artificial Intelligence (AI) applied to MCUs and/or MPUs would require the writing of an entire book, here we will try to give the basics to understand what are the important points to keep in mind and a procedure to follow to arrive at a real application.

The process to follow to arrive at a practical application is summarized by the image below.

Capture Data
Capture a sufficient amount of representative data about the phenomenon that is being modeled.
This usually involves placing sensors on or near the object being monitored in order to record its state and changes over the time. Examples of physical parameters include acceleration, temperature, sound, and visual depending on your application.
ST provides tools that help in data capture and labelling such as our ST BLE Sensor smartphone application which acts as a remote control, see for example:
STWIN SensorTile
SensorTile
SensorTile.box
The cards listed above is equipped with motion and environmental sensors, a microcontroller, SD Card connector and Bluetooth connectivity.

Label Data
(Clean, label Data and Build ANN topology)
Creating an Artificial Neural Network requires labeled data that has been acquired from sensors and pre-processing.
For so-called “supervised learning”, the data sets must be characterized so that the different outputs can be classified correctly.
This classified set is the “ground truth” that will be used to train the ANN and then validate it.
The developer must decide on the type of topology the ANN should have in order to best be able to learn from the data and provide useful output for the target application.
Usually developers employ popular off-the-shelf deep learning frameworks to architect and train Artificial Neural Network topologies.
ST works with a number of Partners who provide Artificial Neural Network engineering services and support with dedicated data scientists and Artificial Neural Network architects.

Train the NN model
Training the ANN (NN) involves passing the data sets through the Neural Network in an iterative manner so that the Network’s outputs can minimize desirable error criteria.
ANN definition, training, and testing is typically performed using off-the-shelf Deep Learning frameworks.
This is usually done on a powerful computing platform, with virtually unlimited memory and computational power, to allow many iterations in a short period of time.
The result of this training is the pre-trained Artificial Neural Network.
The STM32Cube.AI tool offers simple and efficient interoperability with popular Deep Learning training tools widely used by the Artificial Intelligence developer community. The output of these tools can be directly imported into the STM32Cube.AI

STM32 Cube.AI
Convert ANN into optimized code for the STM32 MCU
The next step is to embed the pre-trained ANN into an MCU (optimized code minimizing complexity and memory requirements).
This part is very easy and intuitive thanks to the STM32Cube.AI software tool. The STM32Cube.AI is fully integrated into STM32 software development ecosystem as an extension of the widely used STM32CubeMX tool.
It allows fast, automatic conversion of pre-trained ANNs into optimized code that can run on an MCU. The tool guides users through the selection of the right MCU and provides rapid feedback on the performance of the Neural Network in the chosen MCU, with validation running both on your PC and the target STM32 MCU.
Check out STM Getting Started video.

RUN on STM32
Finally, deploy the ANN embedded in an MCU in your application.
Here ST also makes it easier for designers to quickly prototype their innovative application thanks to integrated software packages – Function Packs.
These packs are end-to-end examples embedding a combination of low-level drivers, middleware libraries and sample applications assembled into a single software package. Developers can easily start from these examples and make modifications to fit their specific application.
Two examples enabled for AI are the Audio and Motion capture & processing Function Packs.
Developers can also get support and exchange ideas in the AI forum of STM32 Community.



STM example

  • STM32 food recognition
    • The examples provided from STM (in the function pack) are food recognition applications.
    • They recognize among 18 classes of common food such as pizza, hamburger, and Caesar salad.
    • The kit is based on: STM32F4DIS-CAM (use the: OV9655 CMOS sensor connected via DCMI bus to STM32H7) + STM32H747I-DISCO (based on STM32H747XIH6)

Interesting links


How to Get Started ?