Go in TiDB 申砾@PingCAP Agenda • About Me • What is TiDB • TiDB Architecture • SQL Layer Internal • Golang in TiDB • Next Step of TiDB About Me • Shen Li (申砾) • Tech Lead of TiDB, VP of Engineering Why Go? • Productivity • Concurrency • Great for network programming • GC • Standard libraries && Tools • Good performance • Quick improvement Go in TiDB • More than 100k lines of Go code and session is hard • Account for large memory allocation • Account for memory consuming operators New in Go1.8 • Better GC • Pauses 50μs • sort.Slice Next Step of TiDB • Switch to gRPC • Document support

Go Practices in TiDB 姚维 PingCAP wink@pingcap.com Agenda ● How to build a stable database ○ Schrodinger-test platform ○ Failpoint injection ○ Goroutine-leak detection ● Optimization ○ Chunk vs interface{} errors.New("mock commit error") // } …. } https://github.com/pingcap/tidb/blob/master/store/tikv/txn.go#L223 The generated gofail code func (txn *tikvTxn) Commit(ctx context.Context) error { if vmockCommitError or: __fp_mockCommitError.BadType(vmockCommitError, "bool"); }; …. } $ gofail enable store/tikv/txn.go Why we need a new failpoint ● Generated code is not readable ● Concurrent testing will use the same

Go runtime related problems in TiDB production environment About me ● Arthur Mao(毛康力), Senior Engineer@PingCAP ● TiDB core developer (top3 contributor) ● GitBook about golang internals (@tiancaiamao) consider priority ● CPU dense workload could affect IO latency Conclusion Part II - Memory control ● Go Runtime ○ Allocated from OS (mmaped) ○ Managed Memory ■ Should the memory be returned to the OS? ○ memory footprint is abnormal ● The memory available on this node is not too much Description ● The Go Runtime thinks it does not use much memory ● The OS does not release the memory (RSS is high) Investigate

做分布式数据库的程序员 ● 现在能写代码的时间是奢侈品 TiDB 的亲爹之一兼首席客服和新功能的第一个用户 ● 冤有头债有主，SQL 慢了来找我。。。 偶尔玩玩音乐 ● 摇滚乐->实验音乐 Go 的粉丝！！！！ 关于 PingCAP 一个有趣的公司 信仰开源，做分布式数据库的，叫 TiDB（好像还蛮火的） TiDB 是分布式的，PingCAP 这个公司也是分布式的（北京、上海、广州、深圳、成 Trace & Span 例子 Jaeger ● CNCF 毕业项目 ● 实现 OpenTracing 标准 ● UI 友好 ● 多种 Sampling 策略 ● 后端存储分离 ● Go! (图标也好看) Example: Example: Carrier ● Tracing 信息携带者 ○ 解决跨服务调用的 Tracing metadata 序列化和反序列化抽象 ● Network Ready Goroutine Wakeup 4.368ms go tool trace go tool trace ● 优点：好用，好看（UI） ● 缺点：性能损耗太大，不能一直开着 Trace in Go runtime ● go tool trace 的原理是？ Trace 会 Go Runtime 的代码中打桩收集 CPU time，在 Goroutine 开始执行时记录

4 40 Request ID CPU Usage 40 30 20 10 1 2 3 4 CPU Profiling By SQL Request ● rust-demo ● go-demo TopSQL: SQL Tag - Digest select id from t where a=1; select id from t where a=2; select id Request Ta g NG-Monitor CPU Usage By Tag CPU Usage By Tag Another approach to CPU resource bind in Go Goroutine CPU Stats ● Try to collect goroutine runtime information. begin := GetGoroutineStats()
 获取执⾏ SQL 消耗的 cpu stats 信息 Trace ● Use Go trace to collect runtime information. curl http://localhost:10080/debug/pprof/trace\?seconds\=1 --output trace.out go tool trace trace.out ● In Goroutine analysis

of replicas. This can guarantee strong consis- tency, and availability when a minority of replicas go down. To meet the requirements of different disaster tolerance levels, you can configure the geographic label configuration: Since TiKV is deployed across different data centers, if the physical machines go down, the Raft Group might lose three of the default five replicas, which causes the cluster 154 configuration Since multiple instances of TiKV are deployed on a single machine, if the physical machines go down, the Raft Group might lose two of the default three replicas, which causes the cluster unavailability

· · · · · · · · · · · · · · · · · · · · · · · · · · · · · · · · · · · · · · · · · · · · 119 4.3.2 Go · · · · · · · · · · · · · · · · · · · · · · · · · · · · · · · · · · · · · · · · · · · · · · · · · · majority of replicas. This guarantees strong consistency and availabil- ity when a minority of replicas go down. You can configure the geographic location and number of replicas as needed to meet different graph of heap profile. This feature 38 �→ provides the same simple and easy-to-use experience as Go heap �→ profiling. 2.2.1 Feature details 2.2.1.1 Scalability • Support designating and isolating

· · · · · · · · · · · · · · · · · · · · · · · · · · · · · · · · · · · · · · · · · · · · 132 4.3.2 Go · · · · · · · · · · · · · · · · · · · · · · · · · · · · · · · · · · · · · · · · · · · · · · · · · · majority of replicas. This guarantees strong consistency and availabil- ity when a minority of replicas go down. You can configure the geographic location and number of replicas as needed to meet different approximately 30 seconds. 6. After your TiDB Cloud cluster is created, click your cluster name to go to the cluster overview page, and then click Connect in the upper-right corner. A connection dialog

of replicas. This can guarantee strong consis- tency, and availability when a minority of replicas go down. To meet the requirements of different disaster tolerance levels, you can configure the geographic �→ preparing �→ - �→ time �→ Newly added Controls the max- i- mum wait- ing time for the store to go on- line. 74 Configuration file Configuration Change type Description TiCDCenable �→ - �→ tls �→ Serverless cluster will be created in approximately 30 seconds. 4. Click the target cluster name to go to its overview page, and then click Connect in the upper-right corner. A connection dialog box is

· · · · · · · · · · · · · · · · · · · · · · · · · · · · · · · · · · · · · · · · · · · · 153 4.3.2 Go · · · · · · · · · · · · · · · · · · · · · · · · · · · · · · · · · · · · · · · · · · · · · · · · · · majority of replicas. This guarantees strong consistency and availabil- ity when a minority of replicas go down. You can configure the geographic location and number of replicas as needed to meet different approximately 30 seconds. 6. After your TiDB Cloud cluster is created, click your cluster name to go to the cluster overview page, and then click Connect in the upper-right corner. A connection dialog