## +21 ## Combining Co-Routines and Functions into a Job System ## HELMUT HLAVACS 20 21 October 24-29 ## About Myself • Professor for Computer Science • University of Vienna, Austria: founded 1365 Processing) Technical Committee 14 Entertainment Computing ## Creating Game Engines with C++ • Vienna Game Job System + • Graphics API Abstraction Layer + • Vienna Entity Component System + Vienna Type List Library // submit command buffers Helmut Hlavacs - Combining Co-Routines and Functions into a Job System - CppCon 2021 4 / 39 ## Modern Multicore CPUs • N>1 independent

nvidia.  COROUTINE JOB SYSTEM WITHOUT STANDARD LIBRARY Tianyi(Tanki) Zhang tankiistanki tankijong Hi Everyone, Tanki awaitable. We will see them many times in the rest of the talk, and they will have different names in the job system.  Then next customization. ## JOB SYSTEM • Scheduler + User defined workload • Optimize for CPU throughput Okay, finally, we are ready to talk about the job system! What is a job system? Job system is a kind of

to make sure every exit path is covered. The deallocation code is always directly following the allocation code. A couple of other tidbits about error handling: - These primitives give enough expressiveness Test [1/1] test "suspend with no resume"... All 1 tests passed. In the same way that each allocation should have a corresponding free, Each suspend should have a corresponding resume. A suspend block that each Wasm page is 64KB in size. On success, returns previous memory size; on failure, if the allocation fails, returns -1. This function is a low level intrinsic with no safety mechanisms usually useful

spark.SparkContext: Starting job: collect at ExecuteStatement.scala:97 2021-10-28 13:56:27.639 INFO kyuubi.SQLOperationListener: Query [a46ca504-fe3a-4dfb-be1e-19770af8ac4c]: Job 3 started with 1 stages, 1 started with 1 tasks, 1 active stages running 2021-10-28 13:56:27.651 INFO scheduler.DAGScheduler: Job 3 finished: collect at ExecuteStatement.scala:97, took 0.016234 s 2021-10-28 13:56:27.653 INFO kyuubi Managers ☑ Job Manager ☑ Submit New Job ☑ Task Manager ☑ Job Manager ☑ Submit New Job ☑ Task Manager ☑ Job Manager ☑ Submit New Job ☑ Task Manager ☑ Job Manager ☑ Submit New Job ☑ Task Manager ☑ Job Manager

spark.SparkContext: Starting job: collect at ExecuteStatement.scala:97 2021-10-28 13:56:27.639 INFO kyuubi.SQLOperationListener: Query [a46ca504-fe3a-4dfb-be1e-19770af8ac4c]: Job 3 started with 1 stages, 1 started with 1 tasks, 1 active stages running 2021-10-28 13:56:27.651 INFO scheduler.DAGScheduler: Job 3 finished: collect at ExecuteStatement.scala:97, took 0.016234 s 2021-10-28 13:56:27.653 INFO kyuubi Uptime: 1.3 min Scheduling Mode: FIFO Completed Jobs: 4 Event Timeline - Completed Jobs (4) Job Id (Job Group) Job Id (Job Group) Description Description select timestamp '2018-11-17' select timestamp

2020-11-02 20:51:49.625 INFO spark.SparkContext: Starting job: collect at ExecuteStatement.scala:49 2020-11-02 20:51:50.129 INFO scheduler.DAGScheduler: Job 0 finished: collect at ExecuteStatement.scala:49, took Page: 11 Pages. Jump to 1. Show 100 items in a page. GoJob Id (Job Group) ▼DescriptionSubmittedDurationStages: Succeeded/TotalTasks 1.6. User Defaults • 2. The Engine Configuration Guide 2.1. How To Use Spark Dynamic Resource Allocation (DRA) in Kyuubi 2.2. How To Use Spark Adaptive Query Execution (AQE) in Kyuubi # 1. Deploy Kyuubi

2020-11-02 20:51:49.625 INFO spark.SparkContext: Starting job: collect at ExecuteStatement.scala:49 2020-11-02 20:51:50.129 INFO scheduler.DAGScheduler: Job 0 finished: collect at ExecuteStatement.scala:49, took Page: 11 Pages. Jump to 1. Show 100 items in a page. GoJob Id (Job Group) ▼DescriptionSubmittedDurationStages: Succeeded/TotalTasks Configurations ☐ 1.6. User Defaults • The Engine Configuration Guide 1. How To Use Spark Dynamic Resource Allocation (DRA) in Kyuubi 2. How To Use Spark Adaptive Query Execution (AQE) in Kyuubi # 1. Deploy Kyuubi

spark.SparkContext: Starting job: collect at ExecuteStatement.scala:97 2021-10-28 13:56:27.639 INFO kyuubi.SQLOperationListener: Query [a46ca504-fe3a-4dfb-be1e-19770af8ac4c]: Job 3 started with 1 stages, 1 started with 1 tasks, 1 active stages running 2021-10-28 13:56:27.651 INFO scheduler.DAGScheduler: Job 3 finished: collect at ExecuteStatement.scala:97, took 0.016234 s 2021-10-28 13:56:27.653 INFO kyuubi 2021-10-28 13:56:27.674 INFO kyuubi.SQLOperationListener: Query [a46ca504-fe3a-4dfb-be1e-19770af8ac4c]: Job 3 succeeded, 0 active jobs running 2021-10-28 13:56:27.744 INFO operation.ExecuteStatement: Processing

spark.SparkContext: Starting job: collect at ExecuteStatement.scala:97 2021-10-28 13:56:27.639 INFO kyuubi.SQLOperationListener: Query [a46ca504-fe3a-4dfb-be1e-19770af8ac4c]: Job 3 started with 1 stages, 1 started with 1 tasks, 1 active stages running 2021-10-28 13:56:27.651 INFO scheduler.DAGScheduler: Job 3 finished: collect at ExecuteStatement.scala:97, took 0.016234 s 2021-10-28 13:56:27.653 INFO kyuubi 2021-10-28 13:56:27.674 INFO kyuubi.SQLOperationListener: Query [a46ca504-fe3a-4dfb-be1e-19770af8ac4c]: Job 3 succeeded, 0 active jobs running 2021-10-28 13:56:27.744 INFO operation.ExecuteStatement: Processing

o Configurations • The Spark SQL Engine Configuration Guide How To Use Spark Dynamic Resource Allocation (DRA) in Kyuubi How To Use Spark Adaptive Query Execution (AQE) in Kyuubi Solution for Big | |---|---|---|---|---| |kyuubi.batch.application.check.interval|PT5S|The interval to check batch job application information.|duration|1.6.0| |kyuubi.batch.application.starvation.timeout|PT3M|Threshold batch job submission. Note that this rule is for server-side protection defined via administrators to prevent some essential config from tampering. You can also pre-define some config for batch job submission