Poster submission: Modern C++ for Parallelism in High Performance Computing Victor Eijkhout CppCon 2024 Introduction This poster reports on ‘D2D’, a benchmark that explores elegance of expression and ‘mini-application’. The same code has been implemented using a number of different approaches to parallelism. Implementations are discussed with performance results. Relevance C++ is making inroads into Since all outputs are independent, it is perfectly parallelizable. (This is sometimes called ‘data parallelism’.) Therefore, achieved parallel speedup will be an important measured quantity in D2D. 2. One

Safe to use ptr, where ptr == untag_pointer(tagged). */ ExampleBringing parallelism to std::ranges algorithmsISO C++ Parallelism/Concurrency Programming Language (based on Gonzalo’s ISC C++ BoF) C++ Ranges Modules Concurrency++ [Bryce, Gonzalo] Coroutines atomic_ref, barriers, … 2024Bringing parallelism to std::ranges algorithms (P3179) // C++03 template R&& r, Comp comp = {}, Proj proj = {});C++ Parallel Range Algorithms: Unifying Parallelism and Ranges Why Combine Parallelism with Ranges? ● Ranges offer a productive API with opportunities for optimization

build on Signal Trees, easy to use, simply design, powerful toolkit for building concurrency and parallelism.Thank You For listeningThank You And thank you to Neva, Dar, Anni, Andrei and Lime TradingQuestions

std::simd how to express inherent parallelism efficiently via data-parallel types Dr. Matthias Kretz GSI Helmholtz Center for Heavy Ion Research CppCon ’23 @mkretz@floss.social github.com/mattkretzMotivation Example: Image Processing Programming Models Outlook Summary Take-Away #1 std::simd expresses data-parallelism — SIMD and ILP Matthias Kretz CppCon ’23 7 GSI Helmholtz Center for Heavy Ion ResearchMotivation 64 0.25 = 256 times faster The naive benchmark was slow because the code didn’t contain any parallelism: one long dependency chain Matthias Kretz CppCon ’23 9 GSI Helmholtz Center for Heavy Ion ResearchMotivation

anything in computer science under the domain of computer graphics, visualization, concurrency, and parallelism. ● Contact information and more on: www.mshah.io 6What you are going to learn today (1/2) ● our original question--what is concurrency ?Parallelism vs Concurrency (programming context) (1/4) Concurrency is often used interchangeably with parallelism--so let’s separate those two terms. 1. Concurrency resources. 2. Parallelism Definition: Everything happens at once, instantaneously 18Parallelism vs Concurrency (programming context) (2/4) Concurrency is often used interchangeably with parallelism--so let’s

weekend 4 Understanding the design space Concurrency vs parallelism, latency vs throughput Cooperative vs preemptive, IO vs CPU 1 5 Parallelism is not concurrency 6 Kernel threading models 7 1:1 Hardware vs Software multithreading Data parallelism, Task parallelism, Dataflow parallelism 2 13 Hardware-level multithreading ILP - Instruction-level Parallelism 1 CPU, multiple “execution ports” SIMD Data parallelism Parallel for loop - Same instructions on multiple data - OpenMP - Use-cases - Vectors, matrices, multi-dimensional arrays and tensors - Challenges: - Nested parallelism - Splitting

Given a logical dataflow with sources S1, S2, … Sn and rates λ1, λ2, … λn identify the minimum parallelism πi per operator i, such that the physical dataflow can sustain all source rates. S1 S2 λ1 Vasiliki Kalavri | Boston University 2020 20 Optimal parallelism per operator ??? Vasiliki Kalavri | Boston University 2020 20 Optimal parallelism per operator captures upstream operators ??? Vasiliki Optimal parallelism per operator captures upstream operators Aggregated true output rate of operator oj , when oj itself and all upstream ops are deployed with optimal parallelism ??? Vasiliki

. . . . . . . . . . . . . . . . . . . . . . . 70 Chapter 07 Parallelism and Concurrency 71 5 CONTENTS CONTENTS 7.1 Basic of Parallelism . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Library document of Regular Expression 70 CHAPTER 07 PARALLELISM AND CONCURRENCY Chapter 07 Parallelism and Concurrency 7.1 Basic of Parallelism std::thread is used to create an execution thread instance critical_section(int change_v) { static std::mutex mtx; 71 7.2 Mutex and Critical Section CHAPTER 07 PARALLELISM AND CONCURRENCY std::lock_guard lock(mtx); // execute contention works v = change_v;

}); return reduce(par_unseq, z, 0, std::plus()); }Outline - Background (Ranges, Parallelism, Distributed Data Structures) - Distributed Ranges (Concepts) - Implementation (Algorithms Data Structures (Dense and sparse matrices) - Lessons learnedOutline - Background (Ranges, Parallelism, Distributed Data Structures) - Distributed Ranges (Concepts) - Implementation (Algorithms views) - Complex Data Structures (Dense and sparse matrices) - Lessons learnedStandard C++ Parallelism - Data structures - Hold and organize data - Views - Lightweight objects, views of data -

is the whole point of all this!) ● many call this “concurrency” vs parallelism because it only gives you the illusion of parallelism since you’re not running anything simultaneously (i.e., on multiple is the whole point of all this!) ● many call this “concurrency” vs parallelism because it only gives you the illusion of parallelism since you’re not running anything simultaneously (i.e., on multiple multiple CPUs) ● but you still have all the synchronization problems from parallelism!executing code with futures/promises ● only need a single thread to execute code with futures/promises because you never