Data is all you need for fusion N 1int x = 4; callee(x); // do work } #include #include #include "benchmark.h" #include "matrix_lib.h" int main(..){ std::vector a; a.reserve(100); reserve(100); // Initialize other data // data-structures. cblas_sgemm(a.data()...); cblas_sgemm(....); return 0; } High Performance code is about Hardware Matrix Multiply 2int x = 4; callee(x); // "matrix_lib.h" int main(..){ std::vector a; a.reserve(100); // Initialize other data // data-structures. cblas_sgemm(a.data()...); cblas_sgemm(....); return 0; } High Performance code is about Hardware

Back to Basics: Algebraic Data Types I also do C++ training! arthur.j.odwyer@gmail.com Arthur O’Dwyer 2020-09-16Outline ● Why the name “algebraic data types”? [3–18] ○ Memory layout diagrams. Why Questions? 2What do I mean by algebraic types? pair C++98. The original algebraic data type. tuple C++11. optional C++17. variant C++17, with minor tweaks to its constructors The memory layout of pair or tuple is going to be pretty much the same as the layout of a plain old data struct. pair tuple 6 A The compiler will do some padding for alignment, and may

Introduction Persistent Hash Map Persistent Transactional Data Structures Live Demonstration ReferencesIntroduction Persistent Hash Map Persistent Transactional Data Structures Live Demonstration References Overview Transactional Data Structures Design Goals Methodology Performance Results Live Demonstration A Persistent Hash Map for Graph Processing Workloads and a Methodology for Persistent Transactional Data Structures Structures 2IntroductionIntroduction Persistent Hash Map Persistent Transactional Data Structures Live Demonstration References Introduction Persistent Memory ▶ Persistent Memory is positioned as a new tier

widget { rect bounding_box(); void place(rect r); }; struct bitmap { std::array data; int width, height; }; 1 2 3 4 5 6 7 8 9 10 void vertical_center(WidgetContainer const& c) { bounding_box(); 2 void place(rect r); 3 }; 4 5 struct bitmap { 6 std::array data; 7 int width, height; 8 }; 9 10 11 // Magic happens 12 } 13 14 void center_bitmaps() bounding_box(); 2 void place(rect r); 3 }; 4 5 struct bitmap { 6 std::array data; 7 int width, height; 8 }; 9 10 void vertical_center(auto const& c) { 11 // Magic happens

1Can data-oriented-design be improved? Ollivier Roberge Collège Jean-de-Brébeuf 2A brief historic • 1960s • FORTRAN • Lisp • 1980s • OOP • Structured programming • Modules • 2000s • Template • DoD (“Data oriented design”) • Not about cache lines, nor struct layout (at its core) • From wikipedia: ”As a design paradigm, data-oriented-design focuses on optimal transformations of data and focuses ” 4Minimalist definition of DoD 𝐷𝑎𝑡𝑎!"#$"# = 𝐹(𝐷𝑎𝑡𝑎%&$"#) Transformation Input Data Output Data Specific transformation Previous transformation Next transformation 5How DoD is used

Exposing the Games Industry: One Data Structure At A TimeGames are … COMPLICATEDBack in the old days, games were: ● Simple; 2D, 2.5D mostly. ● Single threaded. ● Made from scratch (no game engine) ● Environment Variables. ● Error Handler. ● Job system. ● Resource Registry. ● Reads boot data. ● Listens to kernel message pump.Resource Registry Games are constructed with a wide variety of can reserve up front but what if the size to reserve is unknown?. ○ What happens if the amount of data exceeds the reserved amount? ● Removal of elements that is not the end, shifts elements down to

List-like data structures std::vector std::list std::dequestd::vector C++ version of the array-list data structure Backed by a C-style array Automatically allocates a new backing array when inserting vector { public: // constructor, accessors, ... private: size_t size_; size_t capacity_; T* data_; }; 1 2 3 4 5 6 7 8 9 10std::vector Compact layout - elements stored in contiguous memory Inserting elements at positions other than the end is expensive (O(n))std::list C++ version of the linked-list data structure Elements stored in nodes referencing the next nodestd::list Node Node Node Node Node

Distributed Data Structures Extending to GPUsThis Talk Background: how do we write a program for a supercomputer? Introduce PGAS Model, RDMA Building Remote Pointer Types Building Distributed Data Structures Distributed Data Structures Extending to GPUsThis Talk Background: how do we write a program for a supercomputer? Introduce PGAS Model, RDMA Building Remote Pointer Types Building Distributed Data Structures a supercomputer? Introduce PGAS Model, RDMA Building Remote Pointer Types Building Distributed Data Structures Extending to GPUsWhat This Talk Is Not - A distributed implementation of the STL - A

Multi-GPU, multi-CPU systems require partitioning data - Users must manually split up data amongst GPUs / nodes - High-level mechanisms for data distribution / execution necessary. CPU NIC GPU 1 Tile 0 GPU Tile 1 Tile 0 Xe LinkProject Goals - Offer high-level, standard C++ distributed data structures - Support distributed algorithms - Achieve high performance for both multi-GPU, NUMA Background (Ranges, Parallelism, Distributed Data Structures) - Distributed Ranges (Concepts) - Implementation (Algorithms and views) - Complex Data Structures (Dense and sparse matrices)

Quantifiable Stack/Queue Live Demonstration Design and Implementation of Highly Scalable Quantifiable Data Structures in C++ 2MotivationMotivation Quantifiability Vector Space Entropy Measurement Design and number of atoms in the universe is 1082 Design and Implementation of Highly Scalable Quantifiable Data Structures in C++ 4Motivation Quantifiability Vector Space Entropy Measurement Design and Implementation method call takes effect in real-time order. Design and Implementation of Highly Scalable Quantifiable Data Structures in C++ 5Motivation Quantifiability Vector Space Entropy Measurement Design and Implementation