## CS 591 K1: Data Stream Processing and Analytics Spring 2020 ## 4 /02: Elasticity policies and state migration Vasiliki (Vasia) Kalavri vkalavri@bu.edu Streaming applications are long-running • Workload

of the algorithm. ## Execution policies Most algorithms have overloads that accept execution policies. The standard library algorithms support several execution policies, and the library provides corresponding nth/max_element • Much more efficient than sort • Works for any sortable range - Supports execution policies, comparison functions, concepts/requirements • Hides confusing iterator pair Midpoint available

functions of image pulling and pushing. StratoVirt & virtualization: openEuler supports memory elasticity, huge pages, and enhanced I/O subsystems, and leverages multi-channel concurrency to improve I/O mechanism combine to bring a significant improvement in NVDIMM initialization. Optimized scheduling policies of SCHED_IDLE significantly shorten the scheduling delays of prioritized tasks and reduce interference providing flexible resource scaling capabilities for lightweight workloads. ## New Features - Memory elasticity: StratoVirt allocates and reclaims memory on the real-time memory demand of each workload. The

infrastructure complexity. PowerFlex empowers organizations to move faster by delivering flexibility, elasticity, and simplicity with extraordinary predictable performance for mission-critical workloads, while edge devices, private clouds, public clouds, and vSphere and secures them using global security policies. Use Helm or SUSE Rancher App Catalog to deploy and manage applications across any or all these SUSE Rancher lets you automate processes and applies a consistent set of user access and security policies to all your clusters, no matter where they are running. ## Shared tools and services SUSE Rancher

stream processing 6. Ensure availability despite failures 7. Support distribution and automatic elasticity 8. Offer low-latency ## Building a stream processor ![Image](/uploads/documents/b/6/e/7/b6e7 in-memory|partitioned, virtually unlimited, persisted to backends| |Load management|shedding|backpressure, elasticity| |Fault tolerance|limited support, high availability|full support, exactly-once| ## Summary Today

hashes that were created from private data in previous transactions. - Collection-level endorsement policies Private data collections can now optionally be defined with an endorsement policy that overrides Hyperledger Fabric Model • Blockchain network • Identity. • Membership Service Provider (MSP) • Policies • Peers • Ledger • The Ordering Service • Smart Contracts and Chaincode • Fabric chaincode ordering service to peers on a channel • Peers validate transactions against endorsement policies and enforce the policies - Prior to appending a block, a versioning check is performed to ensure that states

distributed streaming Systems Architecture and design Scheduling and load management Scalability and elasticity Fault-tolerance and guarantees State management Operator semantics Window optimizations Filtering

jpg) ## Cloud Computing Cloud Characteristics Service Goals On-Demand Quick Scaling Rapid elasticity Low Cost Pay per use High Reliable Uncertain downtime High Available ## Fault Oriented Once

Back-pressure  (c) Elasticity ## Selectively drop records: • Temporarily trades-off result accuracy for sustainable performance

Distributed Foreign Data Wrappers • Columnar compressed FDW • SQL on HDFS • High Availability • Elasticity – Break 1-1 mapping of shard to node ![Image](/uploads/documents/5/e/9/9/5e9986560b576f780f906844b3d21eef/p41_1