time window) for functional reasons. 4. Each computation in your Flink topology (framework or user code), as well as each network shuffle, takes time and adds to latency. 5. If the application emits GarbageCollector.G1 Young Generation.Time job-/ taskmana ger The total time spent performing G1 Young Generation garbage collection. Status.JVM.GarbageCollector.G1 Old Generation.Time job-/ taskmana ger The total time spent performing G1 Old Generation garbage collection. caolei – 监控Apache Flink应用程序(入门) 进度和吞吐量监控 – 19 4.13.1.2 Example Dashboard Panel Figure 6: TaskManager

Deliverables • One (1) written report of maximum 5 pages (10%). • Code (including pre-processing, deployment, and testing): (40%) • code deliverables must be accompanied by documentation 8 Vasiliki Kalavri • Real-time statistics, e.g. weather maps • Monitor conditions to adjust resources, e.g. power generation • Monitor energy consumption for billing purposes 22 Vasiliki Kalavri | Boston University 2020

They arrive continuously instead of being available a-priori. • They bear an arrival and/or a generation timestamp. • They are produced by external sources, i.e. the DSMS has no control over their operators, e.g. total traffic from a source every minute packet generation time bytes in packet total bytes this minute minute start or end 16 Vasiliki Kalavri | Boston

"/Users/dianfu/code/src/github/pyflink-faq/testing/test_utils.py", line 122, in␣ ˓→setUp self.t_env = TableEnvironment.create(EnvironmentSettings.in_streaming_mode()) File "/Users/dianfu/code/src/github in in_streaming_mode get_gateway().jvm.EnvironmentSettings.inStreamingMode()) File "/Users/dianfu/code/src/github/pyflink-faq/testing/.venv/lib/python3.8/site- ˓→packages/apache_flink-1.14.4-py3.8-macosx-10 egg/pyflink/java_gateway.py",␣ ˓→line 62, in get_gateway _gateway = launch_gateway() File "/Users/dianfu/code/src/github/pyflink-faq/testing/.venv/lib/python3.8/site- ˓→packages/apache_flink-1.14.4-py3.8-macosx-10

"/Users/dianfu/code/src/github/pyflink-faq/testing/test_utils.py", line 122, in␣ ˓→setUp self.t_env = TableEnvironment.create(EnvironmentSettings.in_streaming_mode()) File "/Users/dianfu/code/src/github in in_streaming_mode get_gateway().jvm.EnvironmentSettings.inStreamingMode()) File "/Users/dianfu/code/src/github/pyflink-faq/testing/.venv/lib/python3.8/site- ˓→packages/apache_flink-1.14.4-py3.8-macosx-10 egg/pyflink/java_gateway.py",␣ ˓→line 62, in get_gateway _gateway = launch_gateway() File "/Users/dianfu/code/src/github/pyflink-faq/testing/.venv/lib/python3.8/site- ˓→packages/apache_flink-1.14.4-py3.8-macosx-10

function is executed in the driver process. 31 / 79 Output Operations (2/4) ▶ What’s wrong with this code? ▶ This requires the connection object to be serialized and sent from the driver to the worker. send(record) // executed at the worker } } 32 / 79 Output Operations (2/4) ▶ What’s wrong with this code? ▶ This requires the connection object to be serialized and sent from the driver to the worker. send(record) // executed at the worker } } 32 / 79 Output Operations (3/4) ▶ What’s wrong with this code? ▶ Creating a connection object has time and resource overheads. ▶ Creating and destroying a connection

Continuous Processing & Streaming Analytics Event-driven Applications ✔ ✔ ✔ 扫码加入社群 与志同道合的码友一起 Code Up 阿里云开发者社区 Apache Flink China 2群 粘贴二维码 谢谢！

• Retries simply replay the output that has been checkpointed, i.e. the user’s non- deterministic code is not re-executed Bloom filters for performance • Maintaining a catalog of all IDs ever seen