recovery semantics, and guarantees Vasiliki Kalavri | Boston University 2020 Today’s topics • High-availability and fault-tolerance in distributed stream processing • Recovery semantics and guarantees positive. Vasiliki Kalavri | Boston University 2020 Further resources • Jeong-Hyon Hwang et al. High-Availability Algorithms for Distributed Stream Processing. (ICDE ’05). • http://cs.brown.edu/research/aurora/hwang

Stream Processing and Analytics Vasiliki (Vasia) Kalavri
 vkalavri@bu.edu Spring 2020 3/31: High-availability & reconfiguration ??? Vasiliki Kalavri | Boston University 2020 • To recover from failures state 2 Checkpointing guards the state from failures, but what about process failure? High-availability ??? Vasiliki Kalavri | Boston University 2020 3 Flink processes ??? Vasiliki Kalavri | Boston keeps metadata about application execution, such as pointers to completed checkpoints. • A high-availability mode migrates the responsibility and metadata for a job to another JobManager in case the

Store • Relational Tables • JSON Documents • X Protocol • X DevAPI Replication & High-Availability • InnoDB Cluster, ClusterSet • Group Replication • Router • Built-in Replication

Retractions & results amendment Reconfiguration & updates Debugging Fault-tolerance & high-availability Vasiliki Kalavri | Boston University 2020 actions, alerts continuous analytics … Building

the query is running might be impractical. • state accumulation and re-partitioning • high-availability and low latency requirements • scheduling overhead Challenges in streaming optimization

in production builds Improves failure diagnostics in checked builds. Provides resilience in high-availability systems.Conclusion • Articulate the contracts for your functions, even those parts that cannot

and then SUSE Rancher server is installed using the Helm package manager for Kubernetes. For high-availability, it is recommended to have a three-node Kubernetes cluster. Ensure that you are aware of the

daemon-reload A definition of High Availability Clusters from Wikipedia: High Availability Clusters High-availability clusters (also known as HA clusters , fail-over clusters or Metroclusters Ac- tive/Active ) given a target and fence device parameters. • crmsh - Advanced command-line interface for High-Availability cluster management in GNU/Linux. • pcs - Pacemaker command line interface and GUI. It permits devices. • Keepalived - Keepalived provides simple and robust facilities for loadbalancing and high-availability to Linux system and Linux based infrastructures. Loadbalancing framework relies on well-known

Communications United Architecture (OPC UA) protocol • Time-Sensitive Networking (TSN) protocol • High-availability Seamless Redundancy (HSR)/Parallel Redundancy Protocol (PRP) • Network Configuration Protocol

you could have a passive primary or an active backup. active This identifies a broker in a high-availability configuration which is accepting remote connections. For example, consider the scenario where that point since it is accepting remote connections. passive This identifies a broker in a high-availability configuration which is not accepting remote connections. For example, consider the scenario