Hlavacs – Combining Co-Routines and Functions into a Job System - CppCon 2021 1 / 39Helmut Hlavacs – Combining Co-Routines and Functions into a Job System - CppCon 2021 2 / 39 About Myself • Professor ComputingHelmut Hlavacs – Combining Co-Routines and Functions into a Job System - CppCon 2021 3 / 39 Creating Game Engines with C++ • Vienna Game Job System + • Graphics API Abstraction Layer + • Vienna Entity Engine 2.0 https://github.com/hlavacs 20Helmut Hlavacs – Combining Co-Routines and Functions into a Job System - CppCon 2021 4 / 39 The Game Loop auto prev = high_resolution_clock::now(); while( !finished()

Tianyi(Tanki) Zhang tankiistanki tankijong COROUTINE JOB SYSTEM WITHOUT STANDARD LIBRARY source code of the system: https://github.com/tankiJong/cpp-coroutine-job-system Hi Everyone, Tanki here. Thanks for coming We will see them many times in the rest of the talk, and they will have different names in the job system. 1314 14 template struct task; task sum(int a, int b) { int result = a customization. 4344 44 JOB SYSTEM • Scheduler + User defined workload • Optimize for CPU throughput Okay, finally, we are ready to talk about the job system! What is a job system? Job system is a kind of

对于 AP 类应用，最大值建议不要超过所有 TiKV 节点的 CPU 核数。 作用域: SESSION | GLOBAL 默认值: 20000 这个变量用来设置 index lookup 操作的 batch 大小，AP 类应用适合较大的值，TP 类应用适合较小的值。 作用域: SESSION | GLOBAL 默认值: 4 这个变量用来设置 index lookup 操作的并发度，AP 类应用适合较大的值，TP scan 操作的并发度，AP 类应用适合较大的值，TP 类应用适合较小的值。 作用域：SESSION | GLOBAL 默认值：25000 这个变量用来设置 index lookup join 操作的 batch 大小，AP 类应用适合较大的值，TP 类应用适合较小的 tidb_index_lookup_size tidb_index_lookup_concurrency tidb_index_lookup_join_concurrency tidb_index_lookup_join_concurrency tidb_hash_join_concurrency tidb_index_serial_scan_concurrency tidb_index_join_batch_size TiDB 专用系统变量和语法 - 42 - 本文档使用 书栈(BookStack.CN) 构建 值。 作用域: SESSION | GLOBAL 默认值: 0 这个变量用来设置是否跳过

satisfaction, and employee happiness 2. one of the best predictors of short lead times was small batch sizes of work b. Agile, Continuous Delivery, and the Three Ways 7 c. The First Way: The Principles context of global goals. ii. REDUCE BATCH SIZES1. Another key component to creating smooth and fast flow is performing work in small batch sizes. 2. Large batch sizes result in skyrocketing levels of Development has worked on is released to production deployment. Like in manufacturing, this large batch release creates sudden, high levels of WIP and massive disruptions to all downstream work centers

Jobs, identified by an id (job_id), are written to the channel. Each task in this simulation reads a job_id, waits for a random amount of time and writes back a tuple of job_id and the simulated time julia> function do_work() for job_id in jobs exec_time = rand() sleep(exec_time) # simulates elapsed time doing actual work # typically performed externally. put!(results, (job_id, exec_time)) end end; errormonitor(@async do_work()) end julia> @elapsed while n > 0 # print out results job_id, exec_time = take!(results) println("$job_id finished in $(round(exec_time; digits=2)) seconds") global n = n - 1 end

Jobs, identified by an id (job_id), are written to the channel. Each task in this simulation reads a job_id, waits for a random amount of time and writes back a tuple of job_id and the simulated time julia> function do_work() for job_id in jobs exec_time = rand() sleep(exec_time) # simulates elapsed time doing actual work # typically performed externally. put!(results, (job_id, exec_time)) end end; errormonitor(@async do_work()) end julia> @elapsed while n > 0 # print out results job_id, exec_time = take!(results) println("$job_id finished in $(round(exec_time; digits=2)) seconds") global n = n - 1 end

A FAST AND RELIABLE AUTOMATED VALIDATION TEST SUITE i. Slow and periodic feedback kills – larger batch sizes compound effort needed to identify causes ii. Unit tests – test a single method, class, or costs, 140% increase in programs under development, 78% reduction in cost per program c. SMALL BATCH DEVELOPMENT AND WHAT HAPPENS WHEN WE COMMIT CODE TO TRUNK INFREQUENTLY i. Branching Strategies TRUNK-BASED DEVELOPMENT PRACTICES i. CI and Trunk-based development (TBD) are countermeasures to large batch size ii. TBD enables: 1. Frequent commits 2. Runs of automated tests of the integrated system

frame time • Systems run together • Read and decompress • Smooth player experience DATA BUILDING • Batch execution • Minimise wall clock time • Systems run in isolation • Read/Write and compress • Developer data for global illumination ▶ Process everything in parallel Speaker notesASSUMPTIONS • Built on Job System, with Async IO • Run on single PC with many processor cores • Fast & sizeable memory & storage Techniques for running on single computer - many threads - not about clusters Built for async processing - job system and async IO PC used for examples is from 4-5 years ago 16 physical cores - 32 logical cores

e=: Configure an optional CRON job that auto-updates Juliaup if has a value larger than 0. The actual value controls how often the CRON job will run to check for a new Juliaup version version in seconds. The default value is 0, i.e. no CRON job will be created. • --startup-selfupdate=: Configure how often Julia will check for new versions of Juliaup when Julia is started. The Jobs, identified by an id (job_id), are written to the channel. Each task in this simulation reads a job_id, waits for a random amount of time and writes back a tuple of job_id and the simulated time

e=: Configure an optional CRON job that auto-updates Juliaup if has a value larger than 0. The actual value controls how often the CRON job will run to check for a new Juliaup version version in seconds. The default value is 0, i.e. no CRON job will be created. • --startup-selfupdate=: Configure how often Julia will check for new versions of Juliaup when Julia is started. The Jobs, identified by an id (job_id), are written to the channel. Each task in this simulation reads a job_id, waits for a random amount of time and writes back a tuple of job_id and the simulated time