Inc. All rights Reserved. Kubernetes Operator for Massively Parallel Postgres Goutam Tadi (@goutamtadi) Senior Software Engineer, Pivotal Software Inc Email: gp-kubernetes@pivotal.io PGConf India 2019 2019 Agenda ● Intro to Greenplum ● Kubernetes 101 ● Greenplum for Kubernetes ○ Components ■ Greenplum Operator ■ Greenplum Cluster ● Demo Massively Parallel Postgres Greenplum Greenplum Data Managed Clouds 5 Container Orchestration System Kubernetes Greenplum on Kubernetes 101 Kubernetes Master Greenplum on Kubernetes 101 Kubernetes Master kubelet kube-proxy docker Node kubelet kube-proxy

Greenplum on Kubernetes 容器化MPP数据库 AGENDA 云数据库背景 云数据库实现方案 Greenplum on Kubernetes Greenplum Operator 总结 云数据库背景 云数据库背景 ● 资源变化 ○ 本地资源 → 云 ○ 静态资源 → 弹性需求 ● 数据变化 ○ 内部数据 → 多数据源 ○ 数据规模 → 不易预测 ○ 数据格式 ○ Vertica Eon Mode ● 容器化数据库+Kubernetes ○ Apache Spark ○ CockroachDB ○ Apache HAWQ 云数据库存储方案 ● 块存储 ○ 文件系统接口 ● 对象存储 ○ 成本低 ○ 扩展性强 ○ 访问延迟高 Greenplum on Kubernetes Network Interconnect Standby 容器化Greenplum存储管理 ○ 容器本地存储易失性 ○ 容器外部存储关联性 Kubernetes 容器网络管理 容器资源管理 容器镜像管理 容器调度 容器监控及自 定义操作 容器存储管理 Kubernetes 101 Kubernetes 101 Master组件 Node组件 Kubernetes存储资源 PV ● PersistentVolume ○ 表示一种存储资源，独立容器生命周期

Elastic Block Storage (EBS) 309 Ephemeral 310 AWS Recommendations 310 Master 310 Segments 310 Google Compute Platform (GCP) 310 Virtual Machine Type 310 Compute 310 Memory 310 Network 311 VMware PXF. PXF includes improvements to error handling and error surfacing. PXF bundles newer guava and Google Cloud Storage hadoop2 libraries. The PXF pxf-log4j.properties template file updates a log filter MADlib, GPText, and PostGIS are not yet provided for installation on Ubuntu systems. Greenplum for Kubernetes is not yet provided with this release. The following table lists key known issues in VMware Tanzu

Elastic Block Storage (EBS) 318 Ephemeral 319 AWS Recommendations 319 Master 319 Segments 319 Google Compute Platform (GCP) 319 Virtual Machine Type 319 Compute 319 Memory 319 Network 320 Storage PXF. PXF includes improvements to error handling and error surfacing. PXF bundles newer guava and Google Cloud Storage hadoop2 libraries. The PXF pxf-log4j.properties template file updates a log filter MADlib, GPText, and PostGIS are not yet provided for installation on Ubuntu systems. Greenplum for Kubernetes is not yet provided with this release. The following table lists key known issues in VMware Tanzu

Elastic Block Storage (EBS) 242 Ephemeral 243 AWS Recommendations 243 Master 243 Segments 243 Google Compute Platform (GCP) 243 Virtual Machine Type 243 Compute 243 Memory 243 Network 244 Storage MADlib, GPText, and PostGIS are not yet provided for installation on Ubuntu systems. Greenplum for Kubernetes is not yet provided with this release. The following table lists key known issues in VMware Tanzu availability and correctness in the event of a hardware failure. Kubernetes enables quick recovery from both pod and host failures, and Kubernetes storage services provide a high level of availability for the

Elastic Block Storage (EBS) 313 Ephemeral 314 AWS Recommendations 314 Master 314 Segments 314 Google Compute Platform (GCP) 314 Virtual Machine Type 314 Compute 314 Memory 314 Network 315 Storage PXF. PXF includes improvements to error handling and error surfacing. PXF bundles newer guava and Google Cloud Storage hadoop2 libraries. The PXF pxf-log4j.properties template file updates a log filter MADlib, GPText, and PostGIS are not yet provided for installation on Ubuntu systems. Greenplum for Kubernetes is not yet provided with this release. The following table lists key known issues in VMware Tanzu

•“一切皆可商用”：商业即用型x86 服务器、存储设备、网络 •通过软件很容易将处理能力扩展到 1000s的内核/系统 Greenplum • “黑盒子” • “大铁箱” • 大磁盘 过去Google™ 曾经用来实现信息搜索功能的技术， 现在被Greenplum用于数据仓库 现在的解决方案 12 Greenplum愿景：企业数据集合 13 • 在企业内创建统一的数据运算平台 • 企业所有者可以直接控制其数据实例 1000s商用处理内核 • 将所有处理操作尽量移动到数据附近 计算内核 Greenplu m并行数 据流引擎 对本地磁盘进行直 接的高性能访问 gNet 互连 • 第一个支持互联网级分析技术（由Google普及）的产品 • 采用新的编程模型，在商用硬件上并行处理和执行 • 可以使客户洞察力和数据货币化程度达到前所未有的高度 MapReduce Greenplum MapReduce的优势 最先将SQL和Map-Reduce的功能整合到统一的数 据处理框架中 • 可以帮助企业采集所有数据，并在竞争中获得出色 的洞察力 41 Parallel Dataflow Engine • General purpose parallel-dataflow engine natively executes SQL & MapReduce • Optimized for commodity compute, storage

Recommendations 297 Master 297 VMware Greenplum 6 Documentation VMware by Broadcom 14 Segments 297 Google Compute Platform (GCP) 297 Virtual Machine Type 297 Compute 297 Memory 298 Network 298 Storage PXF. PXF includes improvements to error handling and error surfacing. PXF bundles newer guava and Google Cloud Storage hadoop2 libraries. The PXF pxf-log4j.properties template file updates a log filter 4xlarge instance type. This provides the highest performance given the cost of the AWS resources. Google Compute Platform (GCP) Virtual Machine Type The two most common instance types in GCP are "Standard"

Elastic Block Storage (EBS) 265 Ephemeral 265 AWS Recommendations 265 Master 265 Segments 265 Google Compute Platform (GCP) 266 Virtual Machine Type 266 VMware Greenplum 6 Documentation VMware, Inc PXF. PXF includes improvements to error handling and error surfacing. PXF bundles newer guava and Google Cloud Storage hadoop2 libraries. The PXF pxf-log4j.properties template file updates a log filter 4xlarge instance type. This provides the highest performance given the cost of the AWS resources. Google Compute Platform (GCP) Virtual Machine Type The two most common instance types in GCP are “Standard”

Elastic Block Storage (EBS) 256 Ephemeral 257 AWS Recommendations 257 Master 257 Segments 257 Google Compute Platform (GCP) 258 Virtual Machine Type 258 Compute 258 Memory 258 Network 258 Storage PXF. PXF includes improvements to error handling and error surfacing. PXF bundles newer guava and Google Cloud Storage hadoop2 libraries. The PXF pxf-log4j.properties template file updates a log filter performance given the cost of the AWS resources. VMware Tanzu Greenplum 6 Documentation VMware, Inc 257 Google Compute Platform (GCP) Virtual Machine Type The two most common instance types in GCP are “Standard”