# Poster submission: Modern C++ for Parallelism in High Performance Computing Victor Eijkhout CppCon 2024 ## I ntroduction This poster reports on ‘D2D’, a benchmark that explores elegance of expression ‘mini-application’. The same code has been implemented using a number of different approaches to parallelism. Implementations are discussed with performance results. ## Relevance C++ is making inroads into all outputs are independent, it is perfectly parallelizable. (This is sometimes called 'data parallelism'.) Therefore, achieved parallel speedup will be an important measured quantity in D2D. 2.

### Interesting Upcoming Features from Low Latency, Parallelism and Concurrency From Kona 2023, Tokyo 2024, and St. Louis 2024 PAUL E. MCKENNEY, MAGED MICHAEL & MICHAEL WONG 20 24 September 15 pointer Bringing parallelism to std::ranges algorithms ## I SO C++ Parallelism/Concurrency Programming Language (based on Gonzalo's ISC C++ BoF)  ## Bringing parallelism to std::ranges algorithms (P3179) // C++03 template

## +24 ## Work Contracts Rethinking Task Based Concurrency and Parallelism for Low Latency C++ MICHAEL A MANISCALCO ### “We cannot solve our problems with the same thinking we used when we created them build on Signal Trees, easy to use, simply design, powerful toolkit for building concurrency and parallelism. ## Thank You ## For listening  Work Contracts Rethinking Task Based Concurrency and Parallelism for Low Latency C++ MICHAEL A MANISCALCO ## Resources: Source Code: github.com/buildingcpp/work_contract

## 20 23 October 01 - 06 ## +23 ## : ## MATTHIAS KRETZ # std::�md how to express inherent parallelism efficiently via data-parallel types Dr. Matthias Kretz GSI Helmholtz Center for Heavy Ion Research a|x\_{14} + a|x\_{22} + a| |x\_7 + a|x\_{15} + a|\\cdots| Time Take-Away #1 std::SIMD expresses data-parallelism – SIMD and ILP ## Example: Reaching Peak FLOP Compiler Input: sequential scalar code Compiler for (auto _ : state) { The naive benchmark was slow because the code didn't contain any parallelism: e_read(x); one long dependency chain ||g++ -O3 -DNDEBUG|g++ -O3 -DNDEBUG -march=native| |---|---|---|

to happen before other things Some things can’t happen at the same time synchronization ## PARALLELISM Definition # “Multiple logical threads of execution with no inter-task dependencies” - Daisy EASY Not easy, but a little less hard And it’s really parallelism, not concurrency ### CONCURRENCY MADE EASY Well, less hard. And it’s parallelism, not concurrency ## C++ parallel algorithms Let the C++17 Parallel Algorithms Library and Beyond,” CppCon 2016 David Olsen, “Faster Code Through Parallelism or CPUs and GPUs,” CppCon 2019 # C++ PARALLEL ALGORITHMS Execution policies Add execution policy

## +21 ## Misra Parallelism Safety-critical Guidelines for C++11, 17, Then C++20, 23 ANDREAS WEIS, MICHAEL WONG & ILYA BURYLOV 20 21 October 24-29 ## Staff Engineer at Woven Planet 📸 / 🎧 ComicSansMS not safe High Integrity C++ Coding Standard Version 4.0, Programming Research Ltd, 2013 • Some parallelism Software Engineering Institute CERT C/C++ Coding Standard, Software Engineering Institute Division deep analysis of 81 rules • Started in 2019 at a MISRA meeting • Why are there no rules for parallelism in MISRA? • 2019-2021: Phase 1 complete • Reviewed 81 rules pulled from C++CG • HIC++ • REphrase

\ldots S_{n} $ and rates $ \lambda_{1}, \lambda_{2}, \ldots \lambda_{n} $ identify the minimum parallelism $ \pi_{i} $ per operator i, such that the physical dataflow can sustain all source rates. ## {o}^{k} $$ Optimal parallelism per operator $$ \pi_{i}=\left\lceil\sum_{\forall j:j Optimal parallelism per operator $$ \pi_{i}=\left[\sum_{\forall j:j ## Optimal parallelism per operator $$ \p deployed with optimal parallelism ## Optimal parallelism per operator $$ \pi

weekend ## Understanding the design space Concurrency vs parallelism, latency vs throughput Cooperative vs preemptive, IO vs CPU ## Parallelism is not concurrence Concurrent = Two Queues One Coffee Machine Hardware vs Software multithreading Data parallelism, Task parallelism, Dataflow parallelism ## Hardware-level multithreading ILP - Instruction-level Parallelism 1 CPU, multiple "execution ports" Data parallelism ## Parallel for loop Same instructions on multiple data OpenMP - Use-cases - Vectors, matrices, multi-dimensional arrays and tensors ## - Challenges: - Nested parallelism - Splitting

anything in computer science under the domain of computer graphics, visualization, concurrency, and parallelism. • Contact information and more on: www.mshah.io ## What you are going to learn today (1/2) (So eb71996fa476915c1f1c2383f3c/p17_2.jpg) ## Parallelism vs Concurrency (programming context) (1/4) Concurrency is often used interchangeably with parallelism--so let’s separate those two terms. 1. Concurrency resources. 2. Parallelism Definition: Everything happens at once, instantaneously ## Parallelism vs Concurrency (programming context) (2/4) Concurrency is often used interchangeably with parallelism--so let’s

but fails it immediately. ## Reconfiguration with Savepoints ## Reconfiguration cases • Change parallelism • scale out to process increased load • scale in to save resources • Fix bugs or change business Savepoint • To decrease or increase the parallelism of an application: • Take a savepoint • Cancel the application • Restart it with an adjusted parallelism • The state is automatically redistributed 108e/p29_1.jpg) key ∈ [0, 19] 1. hash(key) = key (identity) 2. subtask(key) = hash(key) % parallelism  ## Can we do