Stream Processing and Analytics Vasiliki (Vasia) Kalavri
 vkalavri@bu.edu Spring 2020 4/16: Skew mitigation ??? Vasiliki Kalavri | Boston University 2020 Key partitioning 2 w2 w1 w3 round-robin keys cause imbalance w2 w1 w3 ??? Vasiliki Kalavri | Boston University 2020 Addressing skew • To address skew, the system needs to track the frequencies of the partitioning key values. • We can then

or change business logic • Optimize execution plan • Change operator placement • skew and straggler mitigation • Migrate to a different cluster or software version 9 Reconfiguration cases ??? changes: external workload and system performance • Identify bottleneck operators, straggler workers, skew • Enumerate scaling actions, predict their effects, and decide which and when to apply • Allocate changes: external workload and system performance • Identify bottleneck operators, straggler workers, skew • Enumerate scaling actions, predict their effects, and decide which and when to apply • Allocate

Vasiliki Kalavri | Boston University 2020 39 • If it compensates for skew, e.g. when there exist popular keys • if there is skew, throughput is bounded by the instance that receives the highest load Carney et. al. Operator Scheduling in a Data Stream Manager. VLDB 2003. • Load balancing and skew mitigation • Muhammad Anis Uddin Nasir et. al. The power of both choices: Practical load balancing for

at full speed. Depending on the specific hardware and host OS, this effect can very significantly skew the CPU usage reported by the OS. The reported CPU usage can be several times higher than what 283 For more information about nested paging, see chapter 10.7, Nested Paging and VPIDs, page 276. Mitigation options: 13.4.1.1 Disable nested paging By disabling nested paging (EPT), the VMM will construct HT. In some cases it may be disabled by default, but please verify as the effectiveness of the mitigation depends on it. The default action taken by VirtualBox is to flush the level 1 data cache when a

at full speed. Depending on the specific hardware and host OS, this effect can very significantly skew the CPU usage reported by the OS. The reported CPU usage can be several times higher than what it information about nested paging, see chapter 10.6, Nested Paging and VPIDs, page 306. The following mitigation options are available. 13.4.1.1 Disable Nested Paging By disabling nested paging (EPT), the VMM HT. In some cases it may be disabled by default, but please verify as the effectiveness of the mitigation depends on it. The default action taken by VirtualBox is to flush the level 1 data cache when a

at full speed. Depending on the specific hardware and host OS, this effect can very significantly skew the CPU usage reported by the OS. The reported CPU usage can be several times higher than what 283 For more information about nested paging, see chapter 10.7, Nested Paging and VPIDs, page 276. Mitigation options: 13.4.1.1 Disable nested paging By disabling nested paging (EPT), the VMM will construct HT. In some cases it may be disabled by default, but please verify as the effectiveness of the mitigation depends on it. The default action taken by VirtualBox is to flush the level 1 data cache when a

at full speed. Depending on the specific hardware and host OS, this effect can very significantly skew the CPU usage reported by the OS. The reported CPU usage can be several times higher than what 282 For more information about nested paging, see chapter 10.7, Nested Paging and VPIDs, page 275. Mitigation options: 13.4.1.1 Disable nested paging By disabling nested paging (EPT), the VMM will construct HT. In some cases it may be disabled by default, but please verify as the effectiveness of the mitigation depends on it. The default action taken by VirtualBox is to flush the level 1 data cache when a

at full speed. Depending on the specific hardware and host OS, this effect can very significantly skew the CPU usage reported by the OS; the reported CPU usage can be several times higher than what it For more information about nested paging, see chapter 10.7, Nested paging and VPIDs, page 256. Mitigation options: 13.4.1.1 Disable nested paging By disabling nested paging (EPT), the VMM will construct HT. In some cases it may be disabled by default, but please verify as the effectiveness of the mitigation depends on it. 281 13 Security guide The default action taken by VirtualBox is to flush the level

at full speed. Depending on the specific hardware and host OS, this effect can very significantly skew the CPU usage reported by the OS. The reported CPU usage can be several times higher than what it information about nested paging, see chapter 10.6, Nested Paging and VPIDs, page 311. The following mitigation options are available. 13.4.1.1 Disable Nested Paging By disabling nested paging (EPT), the VMM HT. In some cases it may be disabled by default, but please verify as the effectiveness of the mitigation depends on it. 332 13 Security Guide The default action taken by VirtualBox is to flush the

at full speed. Depending on the specific hardware and host OS, this effect can very significantly skew the CPU usage reported by the OS. The reported CPU usage can be several times higher than what it information about nested paging, see chapter 10.6, Nested Paging and VPIDs, page 309. The following mitigation options are available. 13.4.1.1 Disable Nested Paging By disabling nested paging (EPT), the VMM HT. In some cases it may be disabled by default, but please verify as the effectiveness of the mitigation depends on it. 330 13 Security Guide The default action taken by VirtualBox is to flush the