Block Poll Enqueue Steal Offload Callback Polling Wait Query • Assume two CPU threads (workers) and one GPU stream • Each worker has: High-priority queue (HPQ): store suspended tasks Block Poll Enqueue Steal Offload Callback Polling Wait Query • Assume two CPU threads (workers) and one GPU stream • Each worker has: High-priority queue (HPQ): store suspended tasks Block Poll Enqueue Steal Offload Callback Polling Wait Query • Assume two CPU threads (workers) and one GPU stream • Each worker has: High-priority queue (HPQ): store suspended tasks

unifex::static_thread_pool workers; ex::sender auto accept_and_process_requests() { return ex::on(low_latency.get_scheduler(), accept_request()) | ex::transfer(workers.get_scheduler()) | ex::then([](auto process_request(request_t); extern unifex::static_thread_pool low_latency; extern unifex::static_thread_pool workers; unifex::task accept_and_process_requests() { while (true) { auto request = co_await co_await ex::on(low_latency.get_scheduler(), accept_request()); co_await ex::on(workers.get_scheduler(), process_request(request)); } } Or write it as a coroutine.17 Schedulers produce senders

addListener(this); reader_.readAll(); } // Python API with ThreadPoolExecutor(max_workers=10) as executor: futures = [] for f in sys.argv[1:]: futures.append(executor calls the callbacks? ■ Inefficient! Do you know why? ○ Main thread releases GIL before blocking, workers acquire GIL before callbackComplexity level 2 : Multiple worker threadsComplexity level 2 : Multiple

Correct handling requires communication – Data movement, or synchronization (same thing) – Stop other workers from waiting forever – Propagate and combine error info from workersNo zero-overhead solution •

Palette (consider color blindness!) ZoneScopedC(Tier::Wait);tracy::SetThreadName(“thread-name”) TBB workers : use task_scheduler_observer class tracy_tbb_observer : public oneapi::tbb::task_scheduler_observer

color names Semantic Palette (consider color blindness!)tracy::SetThreadName(“thread-name”) TBB workers : use task_scheduler_observer class tracy_tbb_observer : public oneapi::tbb::task_scheduler_observer

delegated to “work-engine” composed of queues and threads) • Benefits: Greater scaling as threads/workers/resources are made available • Costs: Producer cannot directly monitor work progress (but indirect

server application with gunicorn. In the following example, we run the server on port 9300 with four workers and a timeout of 5 minutes (300 seconds, for large uploads/downloads, you can also decrease it if error, update Conan to 1.8 or higher. 16.4.7 ERROR: Error while starting Conan Server with multiple workers When running gunicorn to start conan_server in an empty environment: $ gunicorn -b 0.0.0.0:9300 multiple workers are running at same time, it could result in a conflict. To fix this error, you should run: $ conan_server --migrate This command must be executed before to start the workers. It will

server application with gunicorn. In the following example, we run the server on port 9300 with four workers and a timeout of 5 minutes (300 seconds, for large uploads/downloads, you can also decrease it if error, update Conan to 1.8 or higher. 16.4.7 ERROR: Error while starting Conan Server with multiple workers When running gunicorn to start conan_server in an empty environment: $ gunicorn -b 0.0.0.0:9300 multiple workers are running at same time, it could result in a conflict. To fix this error, you should run: $ conan_server --migrate This command must be executed before to start the workers. It will

server application with gunicorn. In the following example, we run the server on port 9300 with four workers and a timeout of 5 minutes (300 seconds, for large uploads/downloads, you can also decrease it if error, update Conan to 1.8 or higher. 16.4.7 ERROR: Error while starting Conan Server with multiple workers When running gunicorn to start conan_server in an empty environment: $ gunicorn -b 0.0.0.0:9300 multiple workers are running at same time, it could result in a conflict. To fix this error, you should run: $ conan_server --migrate This command must be executed before to start the workers. It will