prompting the search for reductions ▶ Growth of distributed software applications: blockchain, distributed file systems, network apps Design and Implementation of Highly Scalable Quantifiable Data Structures

modelsBackground: How to supercompute?What is a Cluster? - A collection of nodes, connected by a network.How do I program one? - Message Passing - processes issue matching send and receive calls Node GPU PCI Bus (or other fabric) NIC - Historically, network comm. was CPU-centric 1) Direct GPU access to Infiniband allows GPU-to-GPU network transfers 2) Fast in-node fabrics like NVLink, Infinity PCI Bus (or other fabric) NIC Data - Historically, network comm. was CPU-centric 1) Direct GPU access to Infiniband allows GPU-to-GPU network transfers 2) Fast in-node fabrics like NVLink, Infinity

⊡ Threads ⊡ Layer Separation ⊡ Network Problems ⊡ External Interfaces ⊡ Simulators ⊡ Logs ⊡ Monitoring⊡ Operating Systems ⊡ Threads ⊡ Layer Separation ⊡ Network Problems ⊡ External Interfaces systems with soft real-time requirements⊡ Operating Systems ⊡ Threads ⊡ Layer Separation ⊡ Network Problems ⊡ External Interfaces ⊡ Simulators ⊡ Logs ⊡ MonitoringMine⊡ Collects meter measurements Loss of data when there were network errors None of the errors were seen in the lab.⊡ Loss of data in big sites (>5000 meters) ⊡ Loss of data when there were network errors None of the errors

during network issues. 16 None of the errors were seen in the lab.THE PROBLEMS ⊡ Memory issues – we didn’t have enough memory. ⊡ Unexplained crashes. ⊡ Occasional data loss during network issues during network issues. 18 None of the errors were seen in the lab.THE PROBLEMS ⊡ Memory issues – we didn’t have enough memory. ⊡ Unexplained crashes. ⊡ Occasional data loss during network issues 65 ⊡ Unexplained crashes. ⊡ Not enough information after crash. ⊡ Occasional data loss during network issues. ⊡ Suddenly, no meters are being detected anymore. Problems:REPRODUCE THE ERROR Unexplained

} }, } 41 openssl.cpsWhat is a component? 42 "network": { … "requires": ["openssl::ssl"] } "mygame": { … "requires": ["network", "engine"] } "engine": { … "requires": ["openssl::crypto"] "requires": ["zlib::zlib"], } network.cps engine.cps openssl.cps ✔What is a component? 43 "network": { … "requires": ["zlib::static"] } "mygame": { … "requires": ["network", "engine"] } "static": "dynamic": { "libs": ["zlib.so"], } network.cps engine.cps zlib.cps X "engine": { … "requires": ["zlib::dynamic"] }What is a component? 44 "network": { … "requires": ["boost::headers"]

def print_packet(packet): print(packet) my_network_library = MyNetworkLibrary() my_network_library.register_callback_on_packet(print_packet) my_network_library.listenAndBlock()std::vector def print_packet(packet): print(packet) my_network_library = MyNetworkLibrary() my_network_library.register_callback_on_packet(print_packet) my_network_library.listenInThread() while True: pr print(get_resource_usage) time.sleep(10)Complexity level 4 : back and forth ● Your network library maintains some connections to external processes (distributed computation) ● Python thread is used

phone, etc, do not have access to GPU. To bridge that gap, micrograd++ let’s any user train a neural network in C++ and ship that to any edge device. II. RELEVANCE The development of micrograd++ is driven The library includes classes for defining layers and neurons, enabling users to construct complex network architec- tures. • Backpropagation: The implementation of backpropa- gation in micrograd++ allows single layer in a neural network, composed of multiple neurons. It supports forward propagation and gradient computation, facilitating the construction and training of complex network architectures. MLP Class:

https://github.com/KDAB/hotspot heaptrack https://github.com/KDE/heaptrack spall https://handmade.network/p/333/spall/ geiger https://github.com/david-grs/geiger Intel IACA https://www.intel.com/conte com/wolfpld/tracy/releases or build it yourself! 1. simply #include whenever needed network (socket) message queues 16 waits for a connection (UDP port 8086) 2. compile TracyClient.cpp memcpy(), malloc() et al. shader compilation sorting, searching, container manipulations, … file, network, logging, … interrupts, user input activity, spurious wake-ups, … 46 Instrument the “usual suspects

https://github.com/KDAB/hotspot heaptrack https://github.com/KDE/heaptrack spall https://handmade.network/p/333/spall/ geiger https://github.com/david-grs/geiger Xpedite https://github.com/morganstanley/Xpedite com/wolfpld/tracy/releases or build it yourself! 1. simply #include whenever needed network (socket) message queues 16 waits for a connection (UDP port 8086) 2. compile TracyClient.cpp memcpy(), malloc() et al. shader compilation sorting, searching, container manipulations, … file, network, logging, … interrupts, user input activity, spurious wake-ups, … 46 Instrument the “usual suspects

Fmod ● DearImgui Available here!One code exampleWhy is it harder to make a fast-paced online game?Network LatencyIt means we are waiting for other player inputs…Confirm frame and current frameData sent through lock-step For example in Age of Empires. From the article by Paul Bettner: 1500 Archers on a 28.8: Network Programming in Age of Empires and BeyondWe still need to wait for all the player inputs to process ● 1500 Archers on a 28.8: Network Programming in Age of Empires and Beyond, by Paul Bettner: https://www.gamedeveloper.com/programming/1500-archers-on-a-28-8-network-programmi ng-in-age-of-empires-and-beyond