chained-task3），Flink会对进出系统的记录和字节进行计数。在这些metrics中，每个operator输出记录的速率 通常是最直观和最容易理解的。 4.2 关键指标 Metric Scope Description numRecordsOutPerSecond task The number of records this operator/task sends per second. marks(例如因为它没有收到任何基于 watermarks的事件)，这也阻止了下游操作符中的watermarks的进展。 4.6 关键指标 Metric Scope Description currentOutputWatermark operator The last watermark this operator has emitted caolei metrics，您可以监视当前消费者组的消息队列头部的落后程度。Flink可以从大多数source获得基本metrics。 4.10 关键指标 Metric Scope Description records-lag-max user applies to FlinkKafkaConsumer The maximum lag in terms of the number

beneficial? ??? Vasiliki Kalavri | Boston University 2020 • Use equivalence transformation rules if the language allows • selection operations are commutative • theta-join operations are commutative • natural Chromium/25.0.1364.160 Chrome/ 25.0.1364.160 Safari/537.22 Referer: https://www.google.be/ Accept-Language: en-US,en;q=0.8 Accept-Charset: ISO-8859-1,utf-8;q=0.7,*;q=0.3 GET /dumprequest HTTP/1.1 Host: Chromium/25.0.1364.160 Chrome/ 25.0.1364.160 Safari/537.22 Referer: https://www.google.be/ Accept-Language: en-US,en;q=0.8 Accept-Charset: ISO-8859-1,utf-8;q=0.7,*;q=0.3 GET /dumprequest HTTP/1.1 Host:

type, content, timing constraints. • Actions define how to produce results from the matches. Language Types 3 Vasiliki Kalavri | Boston University 2020 Three classes of operators: • relation-to-relation: portions of a stream. • relation-to-stream: create streams through querying tables Declarative language: CQL 4 Vasiliki Kalavri | Boston University 2020 Select IStream(*) From S1 [Rows 5], S2 [Rows τ> whenever tuple s is in R at time τ. 6 Vasiliki Kalavri | Boston University 2020 Imperative language: Aurora SQuAl Queries are represented in graphical representation using boxes and arrows Tumble

Boston University 2020 1. Process events online without storing them 2. Support a high-level language (e.g. StreamSQL) 3. Handle missing, out-of-order, delayed data 4. Guarantee deterministic (on Dataflow Streaming Relational Dataflow Input in-order out-of-order Results approximate exact Language SQL extensions, CQL Java, Scala, Python, SQL Execution centralized distributed Parallelism pipeline

split, query, subscribe, … State operations and types 4 Consider you are designing a state interface. What operations should state support? What state types can you think of? • Count, sum, list Kalavri | Boston University 2020 RocksDB 10 RocksDB is an LSM-tree storage engine with key/value interface, where keys and values are arbitrary byte streams. https://rocksdb.org/ https://www.ververica state 22 • A function can work with operator list state by implementing the ListCheckpointed interface • snapshotState() is invoked when Flink triggers a checkpoint of the stateful function. • restoreState()

window Input and output types must be the same Vasiliki Kalavri | Boston University 2020 public interface AggregateFunction extends Function, Serializable { // create a new accumulator two accumulators and return the result. ACC merge(ACC a, ACC b); } 16 AggregateFunction interface Vasiliki Kalavri | Boston University 2020 val avgTempPerWindow: DataStream[(String, Double)] = event-time watermark public abstract long currentWatermark(); } } 19 ProcessWindowFunction interface Get start and end timestamps Iterate over the window contents Vasiliki Kalavri | Boston University

attributes or meta-data. • Consumers subscribe to events by specifying filters in a subscription language. • Filters define constraints in the form of name-value pairs and basic comparison operators

challenging? 28 Vasiliki Kalavri | Boston University 2020 Using pseudocode (or the programming language of your choice), write a program that reads a stream of integers and computes: 29 1. the maximum