## Scalable Stream Processing - Spark Streaming and Flink Amir H. Payberah payberah@kth.se 05/10/2018 https://id2221kth.github.io ## Data Processing Graph Data Pregel, GraphLab, PowerGraph GraphX Chaos Batch Data MapReduce, Dryad FlumeJava, Spark Structured Data Spark SQL Machine Learning Mliib Tensorflow Streaming Data Storm, SEEP, Naiad, Spark Streaming, Flink, Millwheel, Google Dataflow Issues ▶ Continuous vs. micro-batch processing Record-at-a-Time vs. declarative APIs ▶ Spark streaming ▶ Flink ## Spark Streaming ## ▶ Design issues • Continuous vs. micro-batch processing • Record-at-a-Time

## C++20 ♥ SQL John R Bandela, MD ## Overview SQL C++ Example Implementation techniques fixed_string meta_struct Parsing compile time strings into meta_structures  ## Overview SQL C++ Example Implementation Techniques https://github.com/google/cpp-from-the-sky-down/tree/master/meta_struct_20/cppcon_version ## SQL Probably the highest level mainstream HOPE YOU'RE HAPPY. AND I HOPE YOU'VE LEARNED TO SANITIZE YOUR DATABASE INPUTS. ## Avoiding SQL Injection Attacks SELECT orders.id, name, item, price, discount_code FROM orders JOIN customers ON

# Spark 简介以及与 Hadoop 的对比 ## 1 Spark 简介 ### 1.1 Spark 概述 Spark 是 UC Berkeley AMP lab 所开源的类 Hadoop MapReduce 的通用的并行计算框架，Spark 基于 map reduce 算法实现的分布式计算，拥有 Hadoop MapReduce 所具有的优点；但不同于 MapReduce 的是 Job 中间输出和结果可以保存在内存中，从而不再需要读写 HDFS，因此 Spark 能更好地适用于数据挖掘与机器学习等需要迭代的 map reduce 的算法。 ### 1.2 Spark 核心概念 #### 1.2.1 弹性分布数据集（RDD） RDD 是 Spark 的最基本抽象, 是对分布式内存的抽象使用, 实现了以操作本地集合的方式来操作分布式数据集的抽象实现。RDD 是 Spark 最核心的东西, 它表示已被分区, 不可变的并能够被并行操作的数据集合 RDD 的操作不是马上执行，Spark 在遇到 Transformations 操作时只会记录需要这样的操作，并不会去执行，需要等到有 Actions 操作的时候才会真正启动计算过程进行计算。 2. 操作(Actions) (如 : count, collect, save 等), Actions 操作会返回结果或把 RDD 数据写到存储系统中。Actions 是触发 Spark 启动计算的动因。 ####

provide serverless SQL on Data Warehouses and Lakehouses. Kyuubi builds distributed SQL query engines on top of various kinds of modern computing frameworks, e.g., Apache Spark [https://spark.apache.org/], modern data stack. For example, you can use Kyuubi, Spark and Iceberg [https://iceberg.apache.org/] to build and manage Data Lakehouse with pure SQL for both data processing, e.g. ETL, and online analytics processing(OLAP), e.g. BI. All workloads can be done on one platform, using one copy of data, with one SQL interface. ## A Unified Gateway The Server module plays the role of a unified gateway. The server

pitfalls in Firebird SQL Paul Vinkenoog Version 1.2, 30 June 2020 ## Table of Contents 1. What is NULL? ..... 4 1.1. NULL as a default state ..... 4 2. NULL support in Firebird SQL ..... 6 2.1. Disallowing operations will render the entire expression NULL. This guide explores the behaviour of NULL in Firebird SQL, points out common pitfalls and shows you how to deal safely with expressions that contain NULL or reference to refresh your memory, go to the summary at the end of the guide. ## So — what is it? In SQL, NULL is not a value. It is a state indicating that an item's value is unknown or nonexistent.

## TiDB: HBase分布式事务与SQL实现 ## About me • TiDB & Codis founder • Golang expert • Distributed database developer • Currently, CEO and co-founder of PingCAP liuqi@pingcap.com https://github.com/pingcap/tidb What did they say? “Nothing is hotter than SQL-on-Hadoop, and now SQL-on-HBase is fast approaching equal hotness status” Form HBaseCon 2015 We want more! SQL + Transaction(ACID) ## TiDB Features ## • changing a single line of code in most cases. ## • Focus on OLTP ☐ There are lots of OLAP system( Spark, Presto, Impala...) ## TiDB Features ## • Multiple storage engine support TiDB supports most of

Application Programming Interface 3 1.2 Multi-tenancy 4 1.3 High Availability 4 2 Serverless SQL and More 5 2.1 Ease of Use 5 2.2 Run Anywhere at Any Scale 5 2.3 High Performance 6 3 What's to provide serverless SQL on Data Warehouses and Lakehouses. Kyuubi builds distributed SQL query engines on top of various kinds of modern computing frameworks, e.g., Apache Spark, Flink, Doris, Hive, above to build a modern data stack. For example, you can use Kyuubi, Spark and Iceberg to build and manage Data Lakehouse with pure SQL for both data processing, e.g. ETL, and online analytics processing(OLAP)

## RUST CHINA CONF 2023 第三届中国Rust开发者大会 6.17-6.18 @Shanghai ## 用 egg 孵化你的 SQL 优化器 王润基 RisingWave 内核开发工程师 ## RisingLight ## An Educational OLAP Database System ## RisingLight 查询引擎的整体结构  ## 一 个 SQL 语句优化的例子 SQL Physical Plan  ## 查询优化 ## 185d2/p39_1.jpg) https://github.com/risinglightdb/risinglight https://github.com/risinglightdb/sql-optimizer-labs