Michael A Maniscalco ○ Software Architect and Principal Developer at Lime Trading ○ We develop low latency market data and trading software for use in HFT ● Personal ○ github.com/buildingcpp ■ Work Contracts

in your Modern C++ Embedded Project Distributed Clocks Sync0 Sync0/Sync1 DC Jitter System jitter is significantly less than 1us. ciere.com Part II Philosophy Michael Caisse Using

Frame centric Minimise frame time - fast frame rate Eliminate frame time spikes - remove jitter Lag & jitter break immersion for the player Code which takes milliseconds - IO, decompression, web request

Options Automated Trading Systems team at Optiver, a global market maker company - Worked on low-latency systems for 15 years in Trading and DefenseIntroduction 3 • About me - Leading the Options Automated Optiver, a global market maker company - Worked on low-latency systems for 15 years in Trading and Defense • Today’s talk - Engineering low-latency trading systems - Profiling techniques to find bottlenecks related to being fastBeing fast to get the trades – and to be accurate! 11 Ultra low-latency execution (< 1μs) Low-latency strategy layer (1-10 μs)Order book deep dive “The order book is the heart of any

Things happening in SG14 Making C++ better for game development & other low-latency domains Patrice Roy, CeFTI, Université de Sherbrooke, Collège Lionel-Groulx Patrice.Roy@USherbrooke.ca, Patrice.Roy@clg WG21 and WG23 member (but I missed recent WG23 meetings) • Involved quite a bit with SG14, the low-latency study group • Occasional WG21 secretary • etc. 3Who am I? • Father of five, aged 28 to 10 • I WG21 and WG23 member (but I missed recent WG23 meetings) • Involved quite a bit with SG14, the low-latency study group • Occasional WG21 secretary • etc. 4Things happening in SG14 • The abstract of this

https://github.com/erez-strauss/lockfree_mpmc_queueAbout me ● C++/C development for over 25 years ● Low Latency - Trading platforms ● Worked in both banks and hedge funds ● Github: https://github.com/erez-strauss https://github.com/erez-strauss/lockfree_mpmc_queueLockfree, MPMC Queue - Agenda 1. Trading Platform, Latency 2. Queues Environment 3. Queues Requirement 4. Atomic hardware and C++ Atomic 5. Queues Classification e_mpmc_queue6 Latency Fat Tail - Raw Data Response Time of a System 1. 200 samples 2. us - microseconds 3. %TT - % of the total time7 Latency Fat Tail - Simple Histogram8 Latency Fat Tail - Simple

extern unifex::static_thread_pool low_latency; extern unifex::static_thread_pool workers; ex::sender auto accept_and_process_requests() { return ex::on(low_latency.get_scheduler(), accept_request()) request) { process_request(request); }) | unifex::repeat_effect(); } Accept requests on low-latency threads. Process the requests on the worker threads.16 EXAMPLE: TRANSITIONING EXECUTION CONTEXT unifex::static_thread_pool low_latency; extern unifex::static_thread_pool workers; unifex::task accept_and_process_requests() { while (true) { auto request = co_await ex::on(low_latency.get_scheduler()

multithreaded/lock-free code is hard… • There are many factors to consider: • Measurement: Throughput vs latency? • Workload: Proportion of reads vs writes • Hotness: Does the data fit in cache? • Contention: reclamation to get performance that is always competitive with both? • More work on optimizing for low latency (see the GitHub, there is some preliminary work!) Thanks to Guy Blelloch and Hao Wei for their collaboration collaboration on some of this work43 Daniel Anderson -- danielanderson.net Bonus Content: Latency44 Daniel Anderson -- danielanderson.net void retire(T* p) • Indicate that an object has been removed

= 28 158n 1.31u 8.40u 𝑝 = 56 146n 2.43u 13.3u Lock Free Counter Wait Free Counter Load Latency Load Latency30 Benchmarks • My atomic implementation using the wait-free counter versus the = 28 164n 1.21u 7.65u 𝑝 = 56 153n 2.05u 26.4u Lock Free Counter Wait Free Counter Load Latency Load Latency31 Benchmarks • My atomic implementation using the wait-free counter versus the = 28 188n 1.41u 13.3u 𝑝 = 56 244n 7.46u 86.4u Lock Free Counter Wait Free Counter Load Latency Load Latency32 Benchmarks • Which algorithm is best often depends on the workload • How many reads vs