Leveraging the power of C++ for efficient machine learning on embedded devices Adrian Stanciu adrian.stanciu.pub@gmail.com CppCon, 2023 1 / 50About me ◮ I am a software engineer from Romania ◮ I have data to make predictions 11 / 50Neural network (NN) 13 / 50Convolutional neural network (CNN) ◮ Efficient in image classification ◮ A convolutional layer can apply filters to detect: ◮ Edges ◮ Shapes ◮ 50Conclusions ◮ Code isn’t enough... data matters ◮ More diverse data leads to better models ◮ Building accurate models is an expert job ◮ Running on-device inference is straightforward ◮ Running on-device

library aims to provide a simple yet powerful framework for building and training machine learning models. By leveraging the performance efficiency of C++, micro- grad++ offers a robust solution for integrating for defining layers and neurons, enabling users to construct complex network architec- tures. • Backpropagation: The implementation of backpropa- gation in micrograd++ allows for efficient training of of models through gradient descent. • Gradient Clipping: To prevent the issue of exploding gradients, micrograd++ incorporates gradient clipping, ensuring stable training processes. C. Unique Aspects and

control flow • Resource transfer (state pilfering) • Object lifecycle extension To Implement Adapter Layers (assembling or adapting among subsystems): • Asymmetric type-transform (mapping 1:N or N:1 data types (mapping 1:N or N:1 API calls across API boundaries) • Asymmetric coordination of threads or event models across subsystem boundaries (synchronous or asynchronous) Take care to not abuse this (it will evolved in complexity and size Adaptable: Remains manageable as system is adapted to new domains Efficient: Resources are spent wiselyCharley Bay - charleyb123 at gmail dot com The Roles of Symmetry And

20233 Agenda • Motivating example: Matrix multiply • Where std::linalg fits in linear algebra’s layers • std::linalg builds on the C++ Standard Library • std::linalg builds on the long history of the field7 Linear algebra comes in (abstraction) layers What are the responsibilities of a “linear algebra library”?8 Linear algebra comes in (abstraction) layers What are the responsibilities of a “linear math problems • Statistical inference • Physics simulations9 Linear algebra comes in (abstraction) layers What are the responsibilities of a “linear algebra library”? • Layer -1: Fundamentals • Multidimensional

Introduction Conan Documentation, Release 1.44.1 Installation of packages from servers is also very efficient. Only the necessary binaries for the current platform and configuration are downloaded, not all extracting the relevant artifacts for the relevant configuration, while ignoring the others. It is more efficient to build the logic, whereby the same build can be reused to create different packages: This can breaking new API, aimed for the next major version of app1. So the only alternative to be both efficient and have a robust Continuous Integration of changes in our core “products” is to explictly define

Introduction Conan Documentation, Release 1.45.0 Installation of packages from servers is also very efficient. Only the necessary binaries for the current platform and configuration are downloaded, not all extracting the relevant artifacts for the relevant configuration, while ignoring the others. It is more efficient to build the logic, whereby the same build can be reused to create different packages: This can breaking new API, aimed for the next major version of app1. So the only alternative to be both efficient and have a robust Continuous Integration of changes in our core “products” is to explictly define

Introduction Conan Documentation, Release 1.46.2 Installation of packages from servers is also very efficient. Only the necessary binaries for the current platform and configuration are downloaded, not all extracting the relevant artifacts for the relevant configuration, while ignoring the others. It is more efficient to build the logic, whereby the same build can be reused to create different packages: This can breaking new API, aimed for the next major version of app1. So the only alternative to be both efficient and have a robust Continuous Integration of changes in our core “products” is to explictly define

Introduction Conan Documentation, Release 1.56.0 Installation of packages from servers is also very efficient. Only the necessary binaries for the current platform and configuration are downloaded, not all that use those generators and also that information for new generators like CMakeDeps you need both models living together in the same recipe. See also: Read Using Components and package_info() to learn extracting the relevant artifacts for the relevant configuration, while ignoring the others. It is more efficient to build the logic, whereby the same build can be reused to create different packages: This can

Introduction Conan Documentation, Release 1.47.0 Installation of packages from servers is also very efficient. Only the necessary binaries for the current platform and configuration are downloaded, not all extracting the relevant artifacts for the relevant configuration, while ignoring the others. It is more efficient to build the logic, whereby the same build can be reused to create different packages: This can breaking new API, aimed for the next major version of app1. So the only alternative to be both efficient and have a robust Continuous Integration of changes in our core “products” is to explictly define