. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6 1.1.1.3 Standalone . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7 1.1.1.4 YARN contains its own Python executable files and the installed Python packages. It is useful for local development to create a standalone Python environment and also useful when deploying a PyFlink job to output a path like the following: # /path/to/python/site-packages/pyflink # Check the logging under the log directory ls -lh /path/to/python/site-packages/pyflink/log # You will see the log file as following:

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6 1.1.1.3 Standalone . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7 1.1.1.4 YARN contains its own Python executable files and the installed Python packages. It is useful for local development to create a standalone Python environment and also useful when deploying a PyFlink job to output a path like the following: # /path/to/python/site-packages/pyflink # Check the logging under the log directory ls -lh /path/to/python/site-packages/pyflink/log # You will see the log file as following:

Streaming Dataflow Query Processor DAG & StreamOperator Local Single JVM Cloud GCE, EC2 Cluster Standalone, YARN Runtime Distributed Streaming Dataflow DataStream API Stream Processing DataSet API API & SQL Relational Table API & SQL Relational Local Single JVM Cloud GCE, EC2 Cluster Standalone, YARN DataStream Physical 统一 Operator 抽象 Pull-based operator Push-based operator 算子可自定义读取顺序

ResourceManager, e.g. in a YARN setup • A manual TaskManager re-start or a backup is required in standalone mode • The restart strategy determines how often the JobManager tries to restart the application

Kalavri | Boston University 2020 Implementation • Load shedding is commonly implemented by a standalone component integrated with the stream processor • The load shedder continuously monitors input

improve the overall performance by decreasing the number of task slots per TaskManager (in case of a Standalone setup), by providing more resources to the TaskManager (in case of a containerized setup), or