source and stores it in Spark’s memory for processing. ▶ Three categories of streaming sources: 1. Basic sources directly available in the StreamingContext API, e.g., file systems, socket connections. 2 source and stores it in Spark’s memory for processing. ▶ Three categories of streaming sources: 1. Basic sources directly available in the StreamingContext API, e.g., file systems, socket connections. 2 Kinesis, Twitter. 3. Custom sources, e.g., user-provided sources. 13 / 79 Input Operations - Basic Sources ▶ Socket connection • Creates a DStream from text data received over a TCP socket connection

continuous analytics … Building a stream processor… 8 ? Vasiliki Kalavri | Boston University 2020 Basic Stream Models Vasiliki Kalavri | Boston University 2020 A stream can be viewed as a massive, dynamic previously emitted items 12:01 12:02 12:00 18 32 8 32 32 32 8 72 64 80 base derived Which basic models do base and derived streams correspond to? Vasiliki Kalavri | Boston University 2020 Results migration, out-of-order processing 37 Vasiliki Kalavri | Boston University 2020 • No particular basic stream model (time-series, turnstile…) is imposed by the dataflow execution engine. • The burden

pyflink-docs/build/sphinx/html/index.html in the Browser 1.1 Getting Started This page summarizes the basic steps required to setup and get started with PyFlink. There are live notebooks where you can try introduction to the PyFlink Table API, which is used to help novice users quickly understand the basic usage of PyFlink Table API. You can run the latest version of these examples by yourself in ‘Live

pyflink-docs/build/sphinx/html/index.html in the Browser 1.1 Getting Started This page summarizes the basic steps required to setup and get started with PyFlink. There are live notebooks where you can try introduction to the PyFlink Table API, which is used to help novice users quickly understand the basic usage of PyFlink Table API. You can run the latest version of these examples by yourself in ‘Live

in a subscription language. • Filters define constraints in the form of name-value pairs and basic comparison operators. • Constraints can be logically combined to form complex event patterns.

Relational, … Low latency,
 windowing, aggregations, ... 2 Vasiliki Kalavri | Boston University 2020 Basic API Concept Source Data Stream Operator Data Stream Sink Source Data Set Operator Data Set

via a Flink program. • The keys are ordered according to a user-specified comparator function. Basic operations • Get(key): fetch a single key-value from the DB • Put(key, val): insert a single key-value

write a serializer/deserializer for it) • Flink has a built-in type system which supports: • basic types, i.e., String, Long, Integer, Boolean, Array • composite types: Tuples, POJOs, and Scala case