Go in TiDB 申砾@PingCAP Agenda • About Me • What is TiDB • TiDB Architecture • SQL Layer Internal • Golang in TiDB • Next Step of TiDB About Me • Shen Li (申砾) • Tech Lead of TiDB, VP of Engineering Why Go? • Productivity • Concurrency • Great for network programming • GC • Standard libraries && Tools • Good performance • Quick improvement Go in TiDB • More than 100k lines of Go code and } Memory && GC • Query may touch a huge number of data. • Memory allocation may cost a lot of time. • Put pressure on GC worker. • Degrade the performance of SQL. • OOM sucks! So we should take

Real-Time Unified Data Layers: A New Era for Scalable Analytics, Search, and AI v 1.1Table of Contents Introduction 1. The Interconnection of Analytics, Search, and AI 2. What is a Real-Time Unified Unified Data Layer? 3. Why Do You Need a Real-Time Unified Data Layer? 4. 5.CrateDB: A Modern Real-Time Unified Data Layer1. Introduction Data teams are facing more challenges than ever. As applications data engineering and architecture teams must design systems that not only scale but also deliver real-time access and insights. However, the complexity isn’t just technical—business expectations have grown

Go Practices in TiDB 姚维 PingCAP wink@pingcap.com Agenda ● How to build a stable database ○ Schrodinger-test platform ○ Failpoint injection ○ Goroutine-leak detection ● Optimization ○ Chunk vs interface{} Data Data Data Packet Command Data AST Distributed system testing ● Errors can happen anywhere, any time ● Hardware ○ disk error ○ network error ○ CPU ○ clock ● Software ○ file system ○ network & protocol errors.New("mock commit error") // } …. } https://github.com/pingcap/tidb/blob/master/store/tikv/txn.go#L223 The generated gofail code func (txn *tikvTxn) Commit(ctx context.Context) error { if vmockCommitError

No builds required to add new rules. BUSINESS PROBLEM Our Answer Go Language MongoDB Mgo/Beego If we had a system that could do inser*on. Two Go programs perform all the feed management work WHY GO - DYNAMIC FEED MANAGEMENT The Feed System Is Driven By Data Go supports Data from Offer and Internal Feeds DYNAMIC RULE MANAGEMENT Our Answer Go Language MongoDB Mgo/Beego Building rules is a core piece of the

Go runtime related problems in TiDB production environment About me ● Arthur Mao(毛康力), Senior Engineer@PingCAP ● TiDB core developer (top3 contributor) ● GitBook about golang internals (@tiancaiamao) Investigation ● Network IO is ready => goroutine wake up == 4.3ms ○ Sometime even 10ms+ latency here! ○ The time spend on runtime schedule is not negligible ● When CPU is overload, which goroutine should be given consider priority ● CPU dense workload could affect IO latency Conclusion Part II - Memory control ● Go Runtime ○ Allocated from OS (mmaped) ○ Managed Memory ■ Should the memory be returned to the OS? ○

Go � TiDB ����� mongodb lixia@pingcap.com @���_go About me ��� PingCAP �� 15 ���� PingCAP ���� TiDB ��������� review ������� 12 ������� go ���������� 13 ������������������������ ���� ���� ���@���_go Email: lixia@pingcap.com Agenda mongodb introduction gonzo with memory engine gonzo with TiKV engine TiDB and TiKV Transaction Q & A mongodb introduction mongodb introduction

stars 16,000+ commits Marquee adopters 100+ contributors 700+ Slack members History ● Choosing Go ● TCP, Syscalls, GC, Scheduler 2013 2015 2016 First Kubernetes Production launch Last Sunday.

2. Page 8 of 263. You can customize the menu bar and the list of menu items to display, for this, go to Window -> Customize tab. Perspective -> Menu Visibility The toolbar contains buttons for basic enabled buttons are applicable to the active view or editor. You can customize the toolbar, for this, go to tab. Window -> Customize Perspective -> Tool Bar Visibility You can hide or show the toolbar several views and editors simultaneously open in the workspace but only one of them can be active at a time. You can change the layout of the workspace by opening and closing views, docking them in different

2. Page 8 of 263. You can customize the menu bar and the list of menu items to display, for this, go to Window -> Customize tab. Perspective -> Menu Visibility The toolbar contains buttons for basic enabled buttons are applicable to the active view or editor. You can customize the toolbar, for this, go to tab. Window -> Customize Perspective -> Tool Bar Visibility You can hide or show the toolbar several views and editors simultaneously open in the workspace but only one of them can be active at a time. You can change the layout of the workspace by opening and closing views, docking them in different

1. Page 9 of 281. You can customize the menu bar and the list of menu items to display, for this, go to Window -> Customize tab. Perspective -> Menu Visibility The toolbar contains buttons for basic enabled buttons are applicable to the active view or editor. You can customize the toolbar, for this, go to tab. Window -> Customize Perspective -> Tool Bar Visibility You can hide or show the toolbar several views and editors simultaneously open in the workspace but only one of them can be active at a time. You can change the layout of the workspace by opening and closing views, docking them in different