Inc. All rights Reserved. Kubernetes Operator for Massively Parallel Postgres Goutam Tadi (@goutamtadi) Senior Software Engineer, Pivotal Software Inc Email: gp-kubernetes@pivotal.io PGConf India 2019 2019 Agenda ● Intro to Greenplum ● Kubernetes 101 ● Greenplum for Kubernetes ○ Components ■ Greenplum Operator ■ Greenplum Cluster ● Demo Massively Parallel Postgres Greenplum Greenplum Data Managed Clouds 5 Container Orchestration System Kubernetes Greenplum on Kubernetes 101 Kubernetes Master Greenplum on Kubernetes 101 Kubernetes Master kubelet kube-proxy docker Node kubelet kube-proxy

Greenplum on Kubernetes 容器化MPP数据库 AGENDA 云数据库背景 云数据库实现方案 Greenplum on Kubernetes Greenplum Operator 总结 云数据库背景 云数据库背景 ● 资源变化 ○ 本地资源 → 云 ○ 静态资源 → 弹性需求 ● 数据变化 ○ 内部数据 → 多数据源 ○ 数据规模 → 不易预测 ○ 数据格式 ○ Vertica Eon Mode ● 容器化数据库+Kubernetes ○ Apache Spark ○ CockroachDB ○ Apache HAWQ 云数据库存储方案 ● 块存储 ○ 文件系统接口 ● 对象存储 ○ 成本低 ○ 扩展性强 ○ 访问延迟高 Greenplum on Kubernetes Network Interconnect Standby 容器化Greenplum存储管理 ○ 容器本地存储易失性 ○ 容器外部存储关联性 Kubernetes 容器网络管理 容器资源管理 容器镜像管理 容器调度 容器监控及自 定义操作 容器存储管理 Kubernetes 101 Kubernetes 101 Master组件 Node组件 Kubernetes存储资源 PV ● PersistentVolume ○ 表示一种存储资源，独立容器生命周期

Upgrade 301 Backing Up and Restoring a Database 305 Completing the Migration 306 Working With Hash Operator Classes in Greenplum 6 307 Enabling iptables (Optional) 308 Enabling iptables (Optional) 0 How Declaring Distribution Keys 735 Custom Distribution Key Hash Functions 736 Example Custom Hash Operator Class 736 Choosing the Table Storage Model 738 VMware Greenplum v6.19 Documentation VMware, Inc Expressions 882 Column References 883 Positional Parameters 883 Subscripts 884 Field Selection 884 Operator Invocations 885 Function Calls 885 Aggregate Expressions 885 Limitations of Aggregate Expressions

Upgrade 292 Backing Up and Restoring a Database 296 Completing the Migration 297 Working With Hash Operator Classes in Greenplum 6 297 Enabling iptables (Optional) 299 Enabling iptables (Optional) 0 How Declaring Distribution Keys 726 Custom Distribution Key Hash Functions 727 Example Custom Hash Operator Class 727 Choosing the Table Storage Model 729 VMware Greenplum v6.18 Documentation VMware, Inc Expressions 873 Column References 874 Positional Parameters 874 Subscripts 875 Field Selection 875 Operator Invocations 876 Function Calls 876 Aggregate Expressions 876 Limitations of Aggregate Expressions

Upgrade 226 Backing Up and Restoring a Database 229 Completing the Migration 230 Working With Hash Operator Classes in Greenplum 6 231 Enabling iptables (Optional) 232 Enabling iptables (Optional) 0 How Declaring Distribution Keys 658 Custom Distribution Key Hash Functions 659 Example Custom Hash Operator Class 659 Choosing the Table Storage Model 661 VMware Greenplum v6.17 Documentation VMware, Inc Expressions 805 Column References 806 Positional Parameters 806 Subscripts 807 Field Selection 807 Operator Invocations 808 Function Calls 808 Aggregate Expressions 808 Limitations of Aggregate Expressions

Backup 297 Backing Up and Restoring a Database 300 Completing the Migration 301 Working With Hash Operator Classes in Greenplum 6 301 Enabling iptables (Optional) 303 How to Enable iptables 303 Example Declaring Distribution Keys 724 Custom Distribution Key Hash Functions 725 Example Custom Hash Operator Class 725 Choosing the Table Storage Model 727 Heap Storage 727 Append-Optimized Storage 727 Expressions 874 Column References 875 Positional Parameters 875 Subscripts 876 Field Selection 876 Operator Invocations 877 Function Calls 877 Aggregate Expressions 877 Limitations of Aggregate Expressions

使用基于外部组件的资源组来管理外部组件的 CPU 和内存资源。这种资源组，使 用 cgroup 来管理外部组件的 CPU 和内存的使用总量。 注意：GP 的容器化部署，例如 Greenplum for Kubernetes(GP4K)，可能会创建 一组嵌套的 cgroup 配置来管理系统资源，这可能会影响 GP 的资源组管理 CPU 的使 用率、Core 数量和内存使用量，资源组的资源限制将受到上层资源配额的限制。 下面的这些系统模式在所有的DB中都存在：  pg_catalog模式存储着系统表(System Catalog Table)、内置类型(Type)、 函数(Function)和运算符(Operator)。该模式无论是否在search_path中指 明，都存在search_path中，因为没有这个模式的话，SQL将无法执行，数据库 将无法使用。  information_sche expression operator expression(二目运算符) operator expression(一目前置运算符) expression operator(一目后置运算符) 这里的operator可以是一些符号(例如+、-、*、/、>、<等等)，还可以是AND、 OR、NOT等SQL关键字。例如： OPERATOR(datatype,datatype)

Functions 478 Backing Up and Restoring a Database 479 Completing the Migration 480 Working With Hash Operator Classes in Greenplum 6 480 Migrating from Open Source Greenplum Database to VMware Greenplum 482 Declaring Distribution Keys 563 Custom Distribution Key Hash Functions 564 Example Custom Hash Operator Class 564 Choosing the Table Storage Model 566 Heap Storage 566 Append-Optimized Storage 567 Expressions 793 Column References 794 Positional Parameters 794 Subscripts 794 Field Selection 795 Operator Invocations 795 Function Calls 796 Aggregate Expressions 796 Limitations of Aggregate Expressions

Functions 459 Backing Up and Restoring a Database 460 Completing the Migration 461 Working With Hash Operator Classes in Greenplum 6 461 Migrating from Open Source Greenplum Database to VMware Greenplum 463 Declaring Distribution Keys 534 Custom Distribution Key Hash Functions 535 Example Custom Hash Operator Class 535 Choosing the Table Storage Model 537 Heap Storage 537 Append-Optimized Storage 538 Expressions 764 Column References 765 Positional Parameters 765 Subscripts 765 Field Selection 766 Operator Invocations 766 Function Calls 767 Aggregate Expressions 767 Limitations of Aggregate Expressions

Functions 484 Backing Up and Restoring a Database 485 Completing the Migration 485 Working With Hash Operator Classes in Greenplum 6 486 Migrating from Open Source Greenplum Database to VMware Greenplum 487 Declaring Distribution Keys 558 Custom Distribution Key Hash Functions 558 Example Custom Hash Operator Class 559 Choosing the Table Storage Model 561 Heap Storage 561 Append-Optimized Storage 561 Expressions 785 Column References 785 Positional Parameters 786 Subscripts 786 Field Selection 786 Operator Invocations 787 Function Calls 787 Aggregate Expressions 788 Limitations of Aggregate Expressions