2 TALK OUTLINE Part 1: 1. Goals for the Executors proposal 2. Some simple examples, intro to senders 3. The lifecycle of an async operation 4. Under the hood of a concurrent operation 5. Implementing coroutines Part 2: 1. Structured concurrency 2. Cancellation 3. An extended example3 GOALS FOR THE EXECUTORS PROPOSAL The vision: “An asynchronous analog of the STL” • A full suite of standard async algorithms the demo code including the full-fat example from Kirk Shoop at: https://github.com/ericniebler/executors_demo_code_cppcon_2021.git119 DEMO TIME (with huge shoutout to Kirk Shoop)120 Where to now?121

Design patterns for error handling in C++ programs using parallel algorithms and executors Mark Hoemmen* mhoemmen@stellarscience.com CppCon 2020 * hoʊ’mən, or hœm’mən; he/himWho am I? • > 10 years Uncaught exception in task gets captured – Waiting on result throws passed-along exception • P0443 executors + P1897 asynchronous algorithms – Separate path for handling ancestor task’s uncaught exception Uncaught exception in task gets captured – Waiting on result throws passed-along exception • P0443 executors + P1897 asynchronous algorithms – Separate path for handling ancestor task’s uncaught exception

for computing resources is not the same for those queries. It is better for Spark to release some executors when either the query is lightweight, or the SQL engine is being idled. Tuning You can specify isolation, and sharing. On the other hand, we need to enable Spark’s DRA feature for the engines’ executors’ elastic scaling. The Basics of Dynamic Resource Allocation Spark provides a mechanism to dynamically engine has a backlog of pending tasks, it can request executors via ExecutorAllocationManager. When the engine has executors that become idle, the executors are released, and the occupied resources are given

for computing resources is not the same for those queries. It is better for Spark to release some executors when either the query is lightweight, or the SQL engine is being idled. Tuning You can specify isolation, and sharing. On the other hand, we need to enable Spark’s DRA feature for the engines’ executors’ elastic scaling. The Basics of Dynamic Resource Allocation Spark provides a mechanism to dynamically engine has a backlog of pending tasks, it can request executors via ExecutorAllocationManager. When the engine has executors that become idle, the executors are released, and the occupied resources are given

ApplicationMaster and each executor take. Name Default Meaning spark.executor.instances 1 The number of executors for static allocation spark.executor.cores 1 The number of cores to use Name Default Meaning on for computing resources is not the same for those queries. It is better for Spark to release some executors when either the query is lightweight, or the SQL engine is being idled. Tuning You can specify isolation, and sharing. On the other hand, we need to enable Spark’s DRA feature for the engines’ executors’ elastic scaling. 2.1.1. The Basics of Dynamic Resource Allocation Spark provides a mechanism

ApplicationMaster and each executor take. Name Default Meaning spark.executor.instances 1 The number of executors for static allocation spark.executor.cores 1 The number of cores to use Name Default Meaning on for computing resources is not the same for those queries. It is better for Spark to release some executors when either the query is lightweight, or the SQL engine is being idled. Tuning You can specify isolation, and sharing. On the other hand, we need to enable Spark’s DRA feature for the engines’ executors’ elastic scaling. 1.1. The Basics of Dynamic Resource Allocation Spark provides a mechanism to

for computing resources is not the same for those queries. It is better for Spark to release some executors when either the query is lightweight, or the SQL engine is being idled. Tuning You can specify isolation, and sharing. On the other hand, we need to enable Spark’s DRA feature for the engines’ executors’ elastic scaling. 66 Chapter 6. High Availability Kyuubi, Release 1.4.1-incubating The Basics engine has a backlog of pending tasks, it can request executors via ExecutorAllocationManager. When the engine has executors that become idle, the executors are released, and the occupied resources are given

for computing resources is not the same for those queries. It is better for Spark to release some executors when either the query is lightweight, or the SQL engine is being idled. Tuning You can specify isolation, and sharing. On the other hand, we need to enable Spark’s DRA feature for the engines’ executors’ elastic scaling. 66 Chapter 6. High Availability Kyuubi, Release 1.3.0 The Basics of Dynamic engine has a backlog of pending tasks, it can request executors via ExecutorAllocationManager. When the engine has executors that become idle, the executors are released, and the occupied resources are given

ApplicationMaster and each executor take. Name Default Meaning spark.executor.instances 1 The number of executors for static allocation spark.executor.cores 1 The number of cores to use Name Default Meaning on for computing resources is not the same for those queries. It is better for Spark to release some executors when either the query is lightweight, or the SQL engine is being idled. Tuning You can specify isolation, and sharing. On the other hand, we need to enable Spark’s DRA feature for the engines’ executors’ elastic scaling. 2.1.1. The Basics of Dynamic Resource Allocation Spark provides a mechanism

ApplicationMaster and each executor take. Name Default Meaning spark.executor.instances 1 The number of executors for static allocation spark.executor.cores 1 The number of cores to use Name Default Meaning on for computing resources is not the same for those queries. It is better for Spark to release some executors when either the query is lightweight, or the SQL engine is being idled. Tuning You can specify isolation, and sharing. On the other hand, we need to enable Spark’s DRA feature for the engines’ executors’ elastic scaling. 2.1.1. The Basics of Dynamic Resource Allocation Spark provides a mechanism