A Multithreaded, Transaction-Based Locking Strategy for Containers Bob Steagall CppCon 2020 K E W B C O M P U T I N GCopyright © 2020 Bob Steagall K E W B C O M P U T I N G Overview • Sharing a container timestamp ordering (STO) • Testing the STO-based solution • Summary CppCon 2020 - A Multi-threaded, Transaction-Based Locking Strategy for Containers 2Copyright © 2020 Bob Steagall K E W B C O M P U T I N G Writing multi-threaded applications is easier (?) than ever CppCon 2020 - A Multi-threaded, Transaction-Based Locking Strategy for Containers 3Copyright © 2020 Bob Steagall K E W B C O M P U T I N G

LOCK GUARD EXAMPLE if (transaction.amount < account.balance) { account.balance -= transaction.amount; record(account, transaction); accept_transaction(transaction); } else { account -= overdraft_charge; record(account, overdraft_of(transaction)); notify_user(account, overdraft_message); reject_transaction(transaction); } Needs synchronization https://godbolt.org/z/YhdcjncYoCppCon LOCK GUARD EXAMPLE if (transaction.amount < account.balance) { account.balance -= transaction.amount; record(account, transaction); accept_transaction(transaction); } else { account

updates by default, thanks to two auto-retry mechanisms which blindly re-applied updates when a transaction conflicted. TiDB also supports a select ... for update statement which mostly, but not entirely called a timestamp oracle (TSO, for short) to allocate a monotonically increasing se- quence of transaction timestamps. Data is stored in Bigtable, a horizontally scalable key-value store, which offers atomic the state of the row, includ- ing whether that row is locked by a transaction, or a pointer to that row’s current version. A transaction be- gins by acquiring a start timestamp from the TSO. It then acquires

line flushes and memory fences low by persisting only the transaction descriptor: ▶ Contains the information needed to execute a transaction ▶ Leveraged as redo logs verify consistency after a crash and synchronization using a helping scheme in conjunction with CAS ▶ A transaction will help complete another transaction if a conflict is detected ▶ The transaction status is updated from Active to either Committed or Begin Transaction End Transaction Persist Transaction More Operations ? Execute Data Structure Operation Operation Successful? Commit Transaction Abort Transaction Set Transaction Status

Agent B Agent C Agent Request(Transaction ID) A(application) Trasanctionid（CA SDK support） TOM (who) Create a checklist(action) At 2018-0930(time) 日志输出（Transaction ID） C(application) Trasanctionid（CA checklist(action) At 2018-0930(time) 日志输出 Get the corresponding logs for one time request by transaction ID Request(Transaction ID)Java探针的基本原理 A.class 1 2 3 4 5 8 9 Request Response JVM 6 10 7 Class Loader Cloud Native App POD Agent logfile Proxy Transaction ID Transaction ID Cloud Native App POD Agent logfile Proxy Transaction ID Transaction ID …Commit to Client Success 成就客户卓越Mixer组件功能介绍Mixer的介绍

TIMESTAMP TIMESTAMPADD TIMESTAMPDIFF TINYBLOB (R) TINYINT (R) TINYTEXT (R) TO (R) TRAILING (R) TRANSACTION TRIGGER (R) TRIGGERS TRIM TRUE (R) TRUNCATE UNCOMMITTED UNION (R) UNIQUE (R) UNKNOWN UNLOCK (R) (DML) - 151 - 本文档使用 书栈(BookStack.CN) 构建 事务语句 TiDB 支持分布式事务。涉及到事务的语句包括 Autocommit 变量、 START TRANSACTION / BEGIN 、 COMMIT 以及 ROLLBACK 。 语法： 1. SET autocommit = {0 | 1} 通过设置 autocommit 的值为 1，可以将当前 START TRANSACTION; 4. 5. START TRANSACTION WITH CONSISTENT SNAPSHOT; 上述三条语句都是事务开始语句，效果相同。通过事务开始语句可以显式地开始一个新的事务，如果这个时候当前 Session 正在一个事务中间过程中，会将当前事务提交后，开启一个新的事务。 语法： TiDB 事务语句 自动提交 START TRANSACTION, Begin

responsibility goes higher up the chain Right to the top – effectively transaction is wiped Needs local handling to mitigate transaction wiping Pattern : External libraries, modular functionality expresses responsibility goes higher up the chain Right to the top – effectively transaction wiped Needs local handling to mitigate transaction wiping Pattern : External libraries, modular functionality expresses

Column RD/WR wait wait data • Actual data transfer. • Wider and shorter transaction for DDR4 • Narrower and longer transaction for DDR5Memory 23 ACT Row Column RD/WR wait wait data PRE • Capacitor theoretical limit • Bandwidth vs latencyMemory • DDR architecture – it is only a fraction of bus transaction that is in fact DDR • Row addressing • Column addressing • Generation to generation capacitor

使用Multi Raft支持Dynamic Scale ● 支持类 Percolator 分布式事务 ● 提供 MVCC 支持 ○ RocksDB RocksDB Raft KV MVCC Transaction API RPC Store4 Raft groups RPC RPC Client Store1 TiKV Node1 Region 1 Region 3 ... Store2 TiKV region 2 Raft group for region 3 中间件 [0- 10] [10- 20] [20- 30] [30- 40] SQL Layer Transaction API MVCC NewKV NewSQL - 数据库无限水平扩展的完美解决方案 DB Sharding NewSQL | Ti Project 大数据时代，当单机数据 库容量及处理能力达到瓶

Wire Protocol heartbeat meta TiDB is not a database middleware Sharding middleware TiDB ACID Transaction Support Mostly Elastic Scaling Complex Query (Join, Sub query, GROUP BY) Failover Manual Auto wire protocol TiDB Server TiKV Server Storage Physical Stack Highly layered TiKV API (gRPC) Transaction MVCC Multi-Raft (gRPC) RocksDB Raw KV API Transactional KV API TiKV Node Store 1 Region 1