Leveraging the power of C++ for efficient machine learning on embedded devices Adrian Stanciu adrian.stanciu.pub@gmail.com CppCon, 2023 1 / 50About me ◮ I am a software engineer from Romania ◮ I have Image classification ◮ Hand gesture recognition ◮ Summary ◮ Q&A 4 / 50Motivation 5 / 50Machine Learning (ML) ◮ Subfield of Artificial Inteligence (AI) ◮ Enables computers to learn from data and then consumption ◮ May have real-time performance constraints 7 / 50Machine learning on embedded devices ◮ Alternative to cloud-based machine learning ◮ Advantages: ◮ Real-time processing ◮ Low latency ◮ Reduced bandwidth

micrograd++: A 500 line C++ Machine Learning Library Gautam Sharma Independent Researcher gautamsharma2813@gmail.com Abstract—micrograd++ is a pure C++ machine learning li- brary inspired by Andrej Karpathy’s for building and training machine learning models. By leveraging the performance efficiency of C++, micro- grad++ offers a robust solution for integrating machine learning capabilities directly into C++-based Traditionally, all machine learning libraries are extremely bulky and very hard to integrate as third party dependencies. This aspect scares practitioners to adopt a C++ based machine learning library for prototyping

but have never had a chance to study design patterns and need some resources to help orient them. Learning about design patterns and where to apply them can at the least give you a way to think about how for object-oriented programming 26 ● I really enjoyed this book (as a graphics programmer) for learning design patterns. ○ There’s a free web version here: https://gameprogrammingpatterns.com/ ○ I rendering involves: ■ loading font files, a custom string class, a drawable object class, and a deep hierarchy... ■ Oh, by the way, the text must always be green and uppercase letters.Problem we are

Canada ● Chair of Programming Languages for Standards Council of Canada Chair of WG21 SG19 Machine Learning Chair of WG21 SG14 Games Dev/Low Latency/Financial Trading/Embedded ● Editor: C++ SG5 Transactional Critical Advisory Forum • OpenCL/SYCL Safety Critical • Vulkan Safety Critical • JTC1/SC42 Machine Learning WG3 Trustworthiness • ITC22/SC32 SOTIF WG8 SOTIF, WG13, WG14 • SAE ORAD • UL4600 • RISC-V Safety/Security which is much harder in multithread system • Heterogeneous-> AI/ML safety Stage 1: extensive deep analysis of 81 rules • Started in 2019 at a MISRA meeting • Why are there no rules for parallelism

Canada ● Chair of Programming Languages for Standards Council of Canada Chair of WG21 SG19 Machine Learning Chair of WG21 SG14 Games Dev/Low Latency/Financial Trading/Embedded ● Editor: C++ SG5 Transactional Working Group Members Benchmarks/Books Linear Algebra Libraries Implementations Machine Learning Libraries and Parallel Acceleration Frameworks Research neoSYCL SX-AURORA TSUBASA algorithm Data Productivity Languages Big Data Stack, Stats Lib, Databases Learning Productivity Languages Deep Learning, Linear Alg, ML Implement and Test Algorithm Today’s Supercomputing Development

software in the field of applied digital signal processing for 30 years Member of the C++ committee (learning novice) employed by 3OVERVIEW Modules Foundations C++20 Modules, a short recap Module unit // must go here because reasons export namespace libawesome { // implemented elsewhere int deep_thought(...); } // namespace libawesome 1 2 3 4 5 6 7 8 9 export module awesome.v2; // other stuff int deep_thought(...); // implemented elsewhere int much_deeper_thought(...); } // namespace libawesome 1 2 3 4 5 6 7 8 9 10 11 // OEM 1, traditional header // implementation calls 'deep_thought'

Fastest GameBoy Emulator in Modern C++ 5Building the World’s Fastest GameBoy Emulator in Modern C++ 6 Deep System Understanding Emulators require a strong grasp of hardware, low-level operations, and CPU does it have to be fast? • Bragging rights? • To run on low end hardware • To use for machine learning • TEGLON 33TEGLON? • Tetris for • Gameboy • LOgic • iNnversion Where does the term originate -> In case option 3 is a no 3: come home in the comfort of Jax, me and mom. -> In case it hit deep 4: don’t give a F*** and say idc to stand up for yourself. -> Good to prove yourself 91

Several toy examples to learn I hope this is a talk that you can revisit in the future and do a deep dive with the resources at the endWhat is Concurrency? And the possible motivation to work concurrently s/performance-methods-and-practices -of-directx-11-multithreaded-rendering ● Pretty much everywhere in machine learning... ● Pretty much everywhere you need to increase performance! :) 124C++ std::par and std::par_unseq

previous explanations and code - building on the previous example. This is the recommended approach for learning Conan. • The examples also contain hands-on, fully operational examples with code, aimed to explain file: libhello.a ... hello/0.1: Package '54a3ab9b777a90a13e500dd311d9cd70316e9d55' created Let’s deep a bit more in the package method. The generated package() method is using cmake.install() to copy integrations reference. 7.4 Cross-building examples 7.4.1 Creating a Conan package for a toolchain After learning how to create recipes for tool requires that package applications, we are going to show an example

previous explanations and code - building on the previous example. This is the recommended approach for learning Conan. • The examples also contain hands-on, fully operational examples with code, aimed to explain file: libhello.a ... hello/0.1: Package '54a3ab9b777a90a13e500dd311d9cd70316e9d55' created Let’s deep a bit more in the package method. The generated package() method is using cmake.install() to copy Release 2.5.0 7.4 Cross-building examples 7.4.1 Creating a Conan package for a toolchain After learning how to create recipes for tool requires that package applications, we are going to show an example