Stream Processing and Analytics Vasiliki (Vasia) Kalavri
 vkalavri@bu.edu Spring 2020 4/16: Skew mitigation ??? Vasiliki Kalavri | Boston University 2020 Key partitioning 2 w2 w1 w3 round-robin keys cause imbalance w2 w1 w3 ??? Vasiliki Kalavri | Boston University 2020 Addressing skew • To address skew, the system needs to track the frequencies of the partitioning key values. • We can then

or change business logic • Optimize execution plan • Change operator placement • skew and straggler mitigation • Migrate to a different cluster or software version 9 Reconfiguration cases ??? changes: external workload and system performance • Identify bottleneck operators, straggler workers, skew • Enumerate scaling actions, predict their effects, and decide which and when to apply • Allocate changes: external workload and system performance • Identify bottleneck operators, straggler workers, skew • Enumerate scaling actions, predict their effects, and decide which and when to apply • Allocate

Vasiliki Kalavri | Boston University 2020 39 • If it compensates for skew, e.g. when there exist popular keys • if there is skew, throughput is bounded by the instance that receives the highest load Carney et. al. Operator Scheduling in a Data Stream Manager. VLDB 2003. • Load balancing and skew mitigation • Muhammad Anis Uddin Nasir et. al. The power of both choices: Practical load balancing for

= 30的事件时间窗口将被关闭并计算。 因此，您应该在应用程序中对事件时间敏感的operators(如流程函数和窗口)上监控watermarks。如果当前处理 时间与被称为 even-time skew的watermarks之间的差异非常高，那么它通常意味着可能会出现两种情况。首 先，它可能意味着您只是在处理旧的事件，例如在停机后的追赶期间，或者当您的工作无法继续，而事件正在 排队时。其次，它可 state are very application-specific. Typically, an increasing number of keys, a large event-time skew between different input streams or simply missing state cleanup may cause growing state. • NonHeap after recovering from a downtime. During this time you will see a much higher latency (event-time skew) than usual. A sudden increase in the CPU load might also be attributed to high garbage collection

tasks are connected via virtual channels multiplexed over TCP connections: • In the presence of skew, a single overload channel can cause the slowdown of the entire dataflow… can we do better? 25 blocking excess traffic outside the network to protect it. • This is crucial in the presence of data skew where a single overloaded task could otherwise block the flow of data to all other downstream operator

synopses such as histograms can provide much faster estimates. • Sampling is generally sensitive to skew and outliers. • It is difficult to find a good estimator for some queries: • How can we scale

changes: external workload and system performance • Identify bottleneck operators, straggler workers, skew • Enumerate scaling actions, predict their effects, and decide which and when to apply • Allocate