recovery semantics, and guarantees Vasiliki Kalavri | Boston University 2020 Today’s topics • High-availability and fault-tolerance in distributed stream processing • Recovery semantics and guarantees positive. Vasiliki Kalavri | Boston University 2020 Further resources • Jeong-Hyon Hwang et al. High-Availability Algorithms for Distributed Stream Processing. (ICDE ’05). • http://cs.brown.edu/research/aurora/hwang

Stream Processing and Analytics Vasiliki (Vasia) Kalavri
 vkalavri@bu.edu Spring 2020 3/31: High-availability & reconfiguration ??? Vasiliki Kalavri | Boston University 2020 • To recover from failures state 2 Checkpointing guards the state from failures, but what about process failure? High-availability ??? Vasiliki Kalavri | Boston University 2020 3 Flink processes ??? Vasiliki Kalavri | Boston keeps metadata about application execution, such as pointers to completed checkpoints. • A high-availability mode migrates the responsibility and metadata for a job to another JobManager in case the

and Offline installation ● Private container registry support ● Latest 3 versions of k8s ● High-availability control plane ● Auto-repair Installation and Configuration $ gke-on-prem create cluster --dry-run

Retractions & results amendment Reconfiguration & updates Debugging Fault-tolerance & high-availability Vasiliki Kalavri | Boston University 2020 actions, alerts continuous analytics … Building

the query is running might be impractical. • state accumulation and re-partitioning • high-availability and low latency requirements • scheduling overhead Challenges in streaming optimization

splitting functionality, support for the Kubernetes Service Mesh Interface (SMI), graduates high-availability support out of experimental status, and adds a tremendous list of other improvements, performance

machines, each offering local computation and storage. Rather than rely on hardware to deliver high-availability, the library itself is designed to detect and handle failures at the application layer, so delivering

and then SUSE Rancher server is installed using the Helm package manager for Kubernetes. For high-availability, it is recommended to have a three-node Kubernetes cluster. Ensure that you are aware of the

在该VM上运⾏⽤户的容器。请阅读 Building High-Availability Clusters 以实现多主机VM配置。 kube-apiserver kube-apiserver 暴露Kubernetes的API。它是Kubernetes控制能⼒的前端。它被设计为可⽔平扩展——也就是通过部署 更多实例来实现扩容。详⻅ Building High-Availability Clusters 。 etcd