the external storage. • This mode works for output sinks that can be updated in place, such as a MySQL table. 59 / 79 Output Modes ▶ Three output modes: 1. Append: only the new rows appended to the the external storage. • This mode works for output sinks that can be updated in place, such as a MySQL table. 59 / 79 Output Modes ▶ Three output modes: 1. Append: only the new rows appended to the the external storage. • This mode works for output sinks that can be updated in place, such as a MySQL table. 59 / 79 Structured Streaming Example (1/3) ▶ Assume we receive (id, time, action) events

recovery (at-least-once) • It avoids information loss • The output may contain duplicates • A backup needs to rebuild state of the failed node 8 Vasiliki Kalavri | Boston University 2020 Recovery output may contain duplicates • A backup needs to rebuild state of the failed node • Gap recovery (at-most-once) • It drops data during failure • The backup starts from most recent information Can you see any disadvantage in this approach? Vasiliki Kalavri | Boston University 2020 Upstream Backup Upstream nodes act as backups for their downstream operators by logging tuples in their output

Filter out all compromised passwords? • Remove duplicate tuples on recovery when using upstream backup? The membership problem ??? Vasiliki Kalavri | Boston University 2020 22 What data structure Filter out all compromised passwords? • Remove duplicate tuples on recovery when using upstream backup? The membership problem A hash table requires O(logn) bits per element which might still be

automatic if there is a ResourceManager, e.g. in a YARN setup • A manual TaskManager re-start or a backup is required in standalone mode • The restart strategy determines how often the JobManager tries

factories.DynamicTableFactory’ in the classpath . . . . . . . 26 1.3.4.2 O2: ClassNotFoundException: com.mysql.cj.jdbc.Driver . . . . . . . . . . . . . . 29 1.3.4.3 O3: NoSuchMethodError: org.apache.flink.table dependency management page of official PyFlink documentation. 1.3.4.2 O2: ClassNotFoundException: com.mysql.cj.jdbc.Driver py4j.protocol.Py4JJavaError: An error occurred while calling o13.execute. : org.apache java:575) at java.lang.Thread.run(Thread.java:748) Caused by: java.lang.ClassNotFoundException: com.mysql.cj.jdbc.Driver at java.net.URLClassLoader.findClass(URLClassLoader.java:382) at java.lang.ClassLoader

factories.DynamicTableFactory’ in the classpath . . . . . . . 26 1.3.4.2 O2: ClassNotFoundException: com.mysql.cj.jdbc.Driver . . . . . . . . . . . . . . 29 1.3.4.3 O3: NoSuchMethodError: org.apache.flink.table dependency management page of official PyFlink documentation. 1.3.4.2 O2: ClassNotFoundException: com.mysql.cj.jdbc.Driver py4j.protocol.Py4JJavaError: An error occurred while calling o13.execute. : org.apache java:575) at java.lang.Thread.run(Thread.java:748) Caused by: java.lang.ClassNotFoundException: com.mysql.cj.jdbc.Driver at java.net.URLClassLoader.findClass(URLClassLoader.java:382) at java.lang.ClassLoader

2020 Fault-tolerance approaches recap 3 Vasiliki Kalavri | Boston University 2020 Upstream Backup Upstream nodes act as backups for their downstream operators by logging tuples in their output