AI for Embedded Systems – From high-level Framework to Bare Metal

Modern machine learning and AI development are increasingly reliant on complex ecosystems such as PyTorch, Tensor-Flow, or MATLAB. However, porting solutions developed within these high-level frameworks to embedded devices presents significant practical challenges.

The most immediate issue arises from the disconnect between the abstractions used during development and the strict requirements of embedded deployment, where lightweight, dependency-free C code is essential. Additionally, it is easy to design AI applications that exceed the compute and memory constraints of the target embedded hardware. This challenge becomes even more pressing when targeting a diverse range of devices with varying architectures and capabilities with a single application.

In this talk, also a look at the relation between embedded AI and software-architecture construction will be taken, especial-ly in the context of quality attributes as defined by ISO 25010:2023. We present an automated pipeline that transforms high-level AI models into code optimized specifically for the target hardware. Our approach ensures compatibility by opti-mizing for memory footprint, performing network quantization, and tailoring the implementation to the constraints of the embedded system.

Finally, we will demonstrate how feedback from real-world deployments and insights from the code generation process can be used to provide tight provable safety cages for network outputs and runtimes for the application on the target de-vice.