Boston University 2020 • Ensure result correctness • reconfiguration mechanism often relies on fault-tolerance mechanism • State re-partitioning and migration • minimize communication • keep duration

Processing and Analytics Vasiliki (Vasia) Kalavri
 vkalavri@bu.edu Spring 2020 3/24: Exactly-once fault-tolerance in Apache Flink ??? Vasiliki Kalavri | Boston University 2020 Some slides in this lecture diva-portal.org/smash/get/diva2:1240814/FULLTEXT01.pdf 2 ??? Vasiliki Kalavri | Boston University 2020 Fault-tolerance approaches recap 3 Vasiliki Kalavri | Boston University 2020 Upstream Backup Upstream

requires crossing three layers of networking. If one end is faulty, the IP address cannot detect the fault. File systems are manually mounted on the application layer, and active- active links cannot be automatically multiple TCP/RDMA links for different IP addresses, achieving functions such as multi-path link setup, fault recovery and failover, and load balancing. eNFS: Building an E2E High-Performance and Reliable Distributed implement system functions. • System isolation to prevent impact of failures, such as a crash or fault, to OSs that have high security, reliability, and real- time requirements. • Improved resource scheduling

e>) .() 72 / 79 Fault Tolerance (1/2) ▶ Fault tolerance in Spark • RDD re-computation ▶ Fault tolerance in Storm • Tracks records with unique IDs. • Operators been processed. • Records are dropped from the backup when the have been fully acknowledged. ▶ Fault tolerance in Flink • More coarse-grained approach than Storm. • Based on consistent global snapshots runtime overhead, stateful exactly-once semantics. 73 / 79 Fault Tolerance (1/2) ▶ Fault tolerance in Spark • RDD re-computation ▶ Fault tolerance in Storm • Tracks records with unique IDs. • Operators

discovery X X X X X X X - - X X net.if.in[if,] X X X X X X 1 X - X X r mode ▲ bytes (de- fault) X X X X X 2 X X - X X r packets X X X X X X X - X X r errors X X X X X 2 X X - X X r dropped dropped X X X X - X - - X X r net.if.out[if,] X X X X X X 1 X - X X r mode ▲ bytes (de- fault) X X X X X 2 X X - X X r packets X X X X X X X - X X r errors X X X X X 2 X X - X X r dropped X X X X - - X - - - - - net.if.total[if,] X X X X X X 1 X - X X r mode ▲ bytes (de- fault) X X X X X 2 X X - X X r packets X X X X X X X - X X r errors X X X X X 2 X X - X X r dropped X X X

register download Examples Obfuscating and Packing PyQt Application When Things Go Wrong Segment fault Could not find _pytransform The license.lic generated doesn’t work NameError: name ‘__pyarmor__’ is failed when running xxx.py _pytransform can not be loaded twice Check restrict mode failed Protection Fault: unexpected xxx Warning: code object xxxx isn’t wrapped Error: Try to run unauthorized function Check check_mod_pytransform() check_lib_pytransform() except Exception as e: print("Protection Fault: %s" % e) pytransform.sys.exit(1) protect_pytransform() All the string template {xxx} will

server, UNIX is the only operating system that can consis- tently deliver the necessary performance, fault tolerance and resilience. Zabbix operates on market leading versions. Zabbix is tested on the following installations (up to ten monitored hosts), default parameters are sufficient. However, you should change de- fault parameters to maximize performance of Zabbix. See section [Performance tuning] for more details. You collisions[if] - X X X X - X - - X r net.if.in[if,] X X X X X - X - - X r mode ▲ bytes (de- fault) X X X X X1 - X - - X r packets X X X X X - X - - X r errors X X X X X1 - X - - X r dropped X

userid may be used to re- trieve data. 2. Get- ting CPU uti- liza- tion with de- fault and cus- tom pa- ram- e- ters. Alias=cpu.util:system.cpu.util Alias=cpu.util[*]:system.cpu This al- lows use cpu.util key to get CPU utili- sa- tion per- cent- age with de- fault pa- ram- eters as well as use cpu.util[all, idle, avg15] to get spe- cific data about bix servers in par- allel. No spaces al- lowed. If port is not spec- ified, de- fault port is used. IPv6 ad- dresses must be en- closed in square brack- ets if port for

userid may be used to re- trieve data. 2. Get- ting CPU uti- liza- tion with de- fault and cus- tom pa- ram- e- ters. Alias=cpu.util:system.cpu.util Alias=cpu.util[*]:system.cpu This al- lows use cpu.util key to get CPU utili- sa- tion per- cent- age with de- fault pa- ram- eters as well as use cpu.util[all, idle, avg15] to get spe- cific data about bix servers in par- allel. No spaces al- lowed. If port is not spec- ified, de- fault port is used. IPv6 ad- dresses must be en- closed in square brack- ets if port for

userid may be used to re- trieve data. 2. Get- ting CPU uti- liza- tion with de- fault and cus- tom pa- ram- e- ters. Alias=cpu.util:system.cpu.util Alias=cpu.util[*]:system.cpu This al- lows use cpu.util key to get CPU utili- sa- tion per- cent- age with de- fault pa- ram- eters as well as use cpu.util[all, idle, avg15] to get spe- cific data about bix servers in par- allel. No spaces al- lowed. If port is not spec- ified, de- fault port is used. IPv6 ad- dresses must be en- closed in square brack- ets if port for