© 2018 Percona. 1 Peter Zaitsev Using MySQL for Distributed Database Architectures CEO, Percona PingCAP Infra Meetup, Shanghai, China, May 26, 2018 © 2018 Percona. 2 About Percona Solutions enterprises © 2018 Percona. 3 Presentation Cover Basics Why Going Distributed How to do it © 2018 Percona. 4 Distributed ? MySQL Deployment on More than one System © 2018 Percona. 5 Modern Active Users Possible 15M of Daily Active Users counting time of day skew © 2018 Percona. 8 Distributed Systems Tend To be More Complicated to Develop Against More Complicated to Operate Have

Implementation Principle . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10 Elastic Distributed Implementation . . . . . . . . . . . . . . . . . . . . . . . . . 10 Registry Data Structure . . 58 Common Listener Configuration . . . . . . . . . . . . . . . . . . . . . . . . . . . 58 Distributed Listener Configuration . . . . . . . . . . . . . . . . . . . . . . . . . 58 ii 6.2.4 Event Tracing . . . . . . . . 91 10.6 What should you do if you suspect that ElasticJob has a problem in a distributed envi‐ ronment, but it cannot be reproduced and cannot be debugged in the online environment? 91

ElasticJob 的作业可划分为基于 class 类型和基于 type 类型两种。 Class 类型的作业由开发者直接使用，需要由开发者实现该作业接口实现业务逻辑。典型代表：Simple 类 型、Dataflow 类型。Type 类型的作业只需提供类型名称即可，开发者无需实现该作业接口，而是通过外 置配置的方式使用。典型代表：Script 类型、HTTP 类型。 5.5. 作业开放生态 20 Apache getShardingTotalCount(), getShardingItem() 等方法分别获取分片总数，运行在本作业服务 器的分片序列号等。 ElasticJob 目前提供 Simple、Dataflow 这两种基于 class 的作业类型，并提供 Script、HTTP 这两种基于 type 的作业类型，用户可通过实现 SPI 接口自行扩展作业类型。 简单作业 意为简单实现，未经任何封装的类型。需实现 processData(ShardingContext shardingContext, List data) { // process data // ... } } 流式处理 可通过属性配置 streaming.process 开启或关闭流式处理。 如果开启流式处理，则作业只有在 fetchData 方法的返回值为 null 或集合容量为空时，才停止抓取，否 则作业将一直运行下去；如果关闭流式处

最先将SQL和Map-Reduce的功能整合到统一的数 据处理框架中 • 可以帮助企业采集所有数据，并在竞争中获得出色 的洞察力 41 Parallel Dataflow Engine • General purpose parallel-dataflow engine natively executes SQL & MapReduce • Optimized for commodity compute DAS or FC connected storage Local compute Local storage gNet Interconnect Greenplum Parallel Dataflow Engine gNet Software Interconnect • Supercomputing-based “soft switch” interconnect • Utilizes Highly scalable protocol supports 1000s of nodes gNet Software Interconnect SG (Scatter/Gather) Streaming 1) Scatter Phase • No need for dedicated loading servers • Performance scales with number of

Your Mirroring Configuration and Status 615 Checking Disk Space Usage 615 Checking Sizing of Distributed Databases and Tables 615 Viewing Disk Space Usage for a Database 615 Viewing Disk Space Usage instance with error logging 0 Example 5—TEXT Format on a Hadoop Distributed File Server 807 Example 5—TEXT Format on a Hadoop Distributed File Server 0 Example 6—Multiple files in CSV format with header Configuration Parameters 1563 Dispatch Configuration Parameters 1563 Fault Operation Parameters 1563 Distributed Transaction Management Parameters 1564 Read-Only Parameters 1564 Greenplum Mirroring Parameters

Your Mirroring Configuration and Status 605 Checking Disk Space Usage 606 Checking Sizing of Distributed Databases and Tables 606 Viewing Disk Space Usage for a Database 606 Viewing Disk Space Usage 795 Example 4—Single gpfdist instance with error logging 796 Example 5—TEXT Format on a Hadoop Distributed File Server 796 Example 6—Multiple files in CSV format with header rows 797 Example 7—Readable Configuration Parameters 1575 Dispatch Configuration Parameters 1575 Fault Operation Parameters 1575 Distributed Transaction Management Parameters 1576 Read-Only Parameters 1576 Greenplum Mirroring Parameters

Up VMware vSphere Network 351 Verifying the Distributed Virtual Switch Settings 351 Creating the Distributed Port Groups 352 Configuring the Distributed Port Groups 352 VMware Greenplum 6 Documentation by Broadcom 17 Principles of VMware vSphere Distributed Switch to Uplink Assignment 352 Using Four 100GbE Links 353 Enabling VMware vSphere Distributed Switch (vDS) Health Check 354 Next Steps 355 637 Example 4—Single gpfdist instance with error logging 637 Example 5—TEXT Format on a Hadoop Distributed File Server 638 Example 6—Multiple files in CSV format with header rows 638 Example 7—Readable

334 Verifying the Distributed Virtual Switch Settings 334 Creating the Distributed Port Groups 335 Configuring the Distributed Port Groups 336 Principles of VMware vSphere Distributed Switch to Uplink Uplink Assignment 336 Using Four 100GbE Links 336 Enabling VMware vSphere Distributed Switch (vDS) Health Check 337 Next Steps 338 Setting Up VMware vSphere DRS and HA 338 Enabling DRS 338 Enabling HA 608 Example 4—Single gpfdist instance with error logging 608 Example 5—TEXT Format on a Hadoop Distributed File Server 609 Example 6—Multiple files in CSV format with header rows 609 Example 7—Readable

315 Verifying the Distributed Virtual Switch Settings 315 Creating the Distributed Port Groups 316 Configuring the Distributed Port Groups 317 Principles of VMware vSphere Distributed Switch to Uplink Uplink Assignment 317 Using Four 100GbE Links 317 Enabling VMware vSphere Distributed Switch (vDS) Health Check 318 Next Steps 319 Setting Up VMware vSphere DRS and HA 319 Enabling DRS 319 Enabling HA 631 Example 4—Single gpfdist instance with error logging 631 Example 5—TEXT Format on a Hadoop Distributed File Server 632 Example 6—Multiple files in CSV format with header rows 632 Example 7—Readable

304 Verifying the Distributed Virtual Switch Settings 305 Creating the Distributed Port Groups 305 Configuring the Distributed Port Groups 306 Principles of VMware vSphere Distributed Switch to Uplink Uplink Assignment 306 Using Four 100GbE Links 307 Enabling VMware vSphere Distributed Switch (vDS) Health Check 308 Next Steps 308 VMware Tanzu Greenplum 6 Documentation VMware, Inc 13 Setting Up VMware 608 Example 4—Single gpfdist instance with error logging 608 Example 5—TEXT Format on a Hadoop Distributed File Server 609 Example 6—Multiple files in CSV format with header rows 609 Example 7—Readable