Latency Fat Tail - Pie ChartProducer 2 11 Queue Producer N Consumer 1 Consumer 2 Consumer M time tin tout Queue - Message and time flow Producer 1Lockfree, MPMC Queue - Connecting Application Components Capacity: Bounded / dynamic & memory allocation ● Serialization: Strict global order or relaxed per producer order ● API: single item / multiple items, atomicity-ready or not ● Message size: fixed / dynamic unsigned P{2}; std::atomic prod_sum{0}; std::atomic cons_sum{0}; auto producer = [&]() { for (unsigned x = 0; x < N; ++x) { while (!q.push(x))

net/home/lock-free-algorithms/queues/unbounded-spsc-queue https://www.dpdk.org/ 3● Single producer: one producer (aka writer) thread ● Single consumer: one consumer (aka reader) thread ● Lock-free: it fixed-size buffer as if it were connected end-to-end. The oldest entry is processed first. 4 Single Producer Single Consumer Lock-Free Wait-Free Fifo5 Read message from network Handle message Write

R_len, len_x)( fern::Interval (y, C.C_start, C.C_start + C.C_len, len_y)( fern::Compute( fern::Producer( C (x, y, len x, len y) ), fern::Consumer({ A ( x, 0, A.R len, len y), fern::Consumer( B ( + out.x_len, l fern::Interval (y, out.y_start, out.y_start + out.y_len, l fern::Compute( fern::Producer(Input A Input B Output C Matrix Multiplication 59 fern::Interval (x, C.R_start, C.R_start + C R_len, len_x)( fern::Interval (y, C.C_start, C.C_start + C.C_len, len_y)( fern::Compute( fern::Producer( C (x, y, len x, len y) ), fern::Consumer({ A ( x, 0, A.R len, len y), fern::Consumer( B (

from heap and which retains ownership (caller receives handle or light reference) Source…Sink: Producer…Consumer: Sink Symmetry Restored! Manager type provides options for: • No leaked instances • robustly applied because we know what it solves and how to defend against edge cases) Symmetry: Producer ➔ Consumer: Work items are arbitrarily produced; and each is consumed exactly once • Benefits: robustly applied because we know what it solves and how to defend against edge cases) Symmetry: Producer ➔ Consumer: Work items are arbitrarily produced; and each is consumed exactly once • Benefits:

streamnative.io 统⼀消费模型 • Exclusive • Failover • Shared • Key-Shared streamnative.io 统⼀消费模型 — 订阅 Producer Topic 1 2 3 4 5 6 7 Subscription2 Consumer 1 2 3 4 5 6 7 readEntries Ack/N-Ack Broke BookKeepe 1 Topic (T1) Topic (T1) Topic (T1) Subscription (S1) Subscription (S1) Producer (P1) Consumer (C1) Producer (P3) Producer (P2) Consumer (C2) Beijing Shanghai Guangzhou streamnative.io ⼤数据处理中统⼀的存储的抽象 ⼤数据处理中统⼀的存储的抽象 — Segmented Stream Segment0 Segment1 Segment2 Segment3 Producer Consumer Time Segment 5 Segment4 Segment Readers Segmented
 Stream streamnative.io 基础决定上层：Cloud-Native • State / Scale

tornado.queues module (and the very similar Queue classes in asyncio) implements an asynchronous producer / consumer pattern for coroutines, analogous to the pattern implemented for threads by the Python allow_ipv6: bool | None = None, client_key: str | None = None, client_cert: str | None = None, body_producer: Callable[[Callable[[bytes], None]], Future[None]] | None = None, expect_100_continue: bool = False HTTP request body as a string (byte or unicode; if unicode the utf-8 encoding will be used) • body_producer (collections.abc.Callable) – Callable used for lazy/asynchronous request bodies. It is called with

help solve the readers/writers problemOther popular concurrency patterns ● Producer Consumer pattern ○ Pattern in which ‘producer’ threads generate data ■ (Usually adding to some queue) ○ Consumer threads threads then consume any data from the queue, and process it. ○ https://en.wikipedia.org/wiki/Producer%E2%80%93consumer_problem ● Barrier ○ Allow a fixed number of threads to enter a critical section com/wp/semaphores/ ○ Useful puzzles to reason about common concurrency patterns (reader/writer, producer/consumer, etc.) ● The ‘comet book’ https://pages.cs.wisc.edu/~remzi/OSTEP/ ○ (Free/donation based)

asyncio [https://docs.python.org/3/library/asyncio.html#module-asyncio]) implements an asynchronous producer / consumer pattern for coroutines, analogous to the pattern implemented for threads by the Python org/3/library/stdtypes.html#str] | None [https://docs.python.org/3/library/constants.html#None] = None, body_producer: Callable [https://docs.python.org/3/library/typing.html#typing.Callable][[Callable [https://docs HTTP request body as a string (byte or unicode; if unicode the utf-8 encoding will be used) body_producer (collections.abc.Callable [https://docs.python.org/3/library/collections.abc.html#collections.abc

is raised.mutex + condition_variable ● Whenever you have a “producer” and a “consumer”... ○ ...where the consumer must wait for the producer... ○ ...and production and consumption happen over and over writes, to avoid UB. ○ Maybe use a reader-writer lock (std::shared_mutex) for perf. ● Remember: Producer/consumer? Use mutex + condition_variable. ● Best of all, though: Avoid sharing mutable data between

or coroutines: Julia Tasks allow suspending and resuming computations for I/O, event handling, producer-consumer processes, and similar patterns. Tasks can synchronize through operations like wait and example is the producer-consumer problem, where one complex procedure is generating values and another complex procedure is consuming them. The consumer cannot simply call a producer function to get get a value, because the producer may have more values to generate and so might not yet be ready to return. With tasks, the producer and consumer can both run as long as they need to, passing values back