optimizations • plan translation alternatives • Runtime optimizations • load management, scheduling, state management • Optimization semantics, correctness, profitability Topics covered in this There may exist several ways to execute a computation • query plans, e.g. order of operators • scheduling and placement decisions • different algorithms, e.g. hash-based vs. broadcast join • What does data structures • sorting vs hashing • indexing, pre-fetching • minimize disk access • scheduling Objectives • optimize resource utilization or minimize resources • decrease latency, increase

representing, summarizing, and analyzing data streams Systems Algorithms Architecture and design Scheduling and load management Scalability and elasticity Fault-tolerance and guarantees State management management • Analysis of real-time vehicle locations to improve traffic conditions • Provide real-time scheduling information for public transport • Optimize transport network flow and recommend alternative routes

YARN Apache Hadoop YARN is a cluster resource management framework for managing the resources and scheduling jobs in a Hadoop cluster. It’s supported to submit PyFlink jobs to YARN for execution. Set up

YARN Apache Hadoop YARN is a cluster resource management framework for managing the resources and scheduling jobs in a Hadoop cluster. It’s supported to submit PyFlink jobs to YARN for execution. Set up