Protocol Decode Protocol encode Session Context parser validator Logical Optimize Physical Optimize SQL AST TiDB Executor Logical Plan Physical Plan Distributed Coprocessor Statistics TiKV DDL Worker GC Worker BG Job Worker Schema Manager Privilege Manager Feedback SQL Data Data Data Data Packet Command Data AST Distributed system testing ● Errors can happen anywhere, any time ● Hardware ○ disk error ○ network Errorf("mock failpoint error") } AST Rewrite ● Define a group of marker functions ● Parse imports and prune a source file which does not import failpoint ● Traverse AST to find marker function calls ●

PTS Magical Flag Why fast? Clickhouse Mutation Trouble Shoot ● SQL is too big(max_ast_elements, max_expanded_ast_elements) ● Mutations are stuck (KILL MUTATION) ● Zookeeper OOM because of SQL length

PTS Magical Flag Why fast? Clickhouse Mutation Trouble Shoot ● SQL is too big(max_ast_elements, max_expanded_ast_elements) ● Mutations are stuck (KILL MUTATION) ● Zookeeper OOM because of SQL length

the following SQL: SELECT id, name FROM t_user WHERE status = 'ACTIVE' AND age > 18 Its parsing AST (Abstract Syntax Tree) is this: To better understand, the Token of keywords in abstract syntax tree adopts ANTLR as a generator for the SQL parsing engine, and uses Visit to obtain SQL Statement from AST. Starting from version 5.0.x, the architecture of the parsing engine has been refactored. At the same directly obtain the parsing results of the same SQL to improve parsing effi‐ ciency by putting the AST obtained from the first parsing into the cache. Therefore, we recommend that users adopt PreparedStatement

code (op_que_events) Int32 Database handle Buffer Events parameter buffer Int32 Ast function address Int32 Ast parameters function address Int32 Local event id Server Generic response — where

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 44 Parser and AST . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 44 AST . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . to know to contribute to specific parts of Vitess Parser and AST description: Everything you need to know to contribute to parser and AST of Vitess Vitess houses its own SQL parser that is used to parse

ShardingSphere document, v5.0.0 6.1.2 SQLVisitorFacade SPI Name Description SQLVisitorFacade SQL AST visitor facade Implementation Class Description MySQLS tatementSQLVisitorFacade SQL visitor of statement status = 'ACTIVE' AND age > 18 7.1. Sharding 235 Apache ShardingSphere document, v5.0.0 Its parsing AST (Abstract Syntax Tree) is this: To better understand, the Token of keywords in abstract syntax tree adopts ANTLR as a generator for the SQL parsing engine, and uses Visit to obtain SQL Statement from AST. Starting from version 5.0.x, the architecture of the parsing engine has been refactored. At the same

SQL parser facade for openGauss 6.5.2 SQLVisitorFacade SPI Name Description SQLVisitorFacade SQL AST visitor facade Implementation Class Description MySQLS tatementSQLVisitorFacade SQL visitor of statement the following SQL: SELECT id, name FROM t_user WHERE status = 'ACTIVE' AND age > 18 Its parsing AST (Abstract Syntax Tree) is this: 7.2. Sharding 245 Apache ShardingSphere document, v5.1.1 To better adopts ANTLR as a generator for the SQL parsing engine, and uses Visit to obtain SQL Statement from AST. Starting from version 5.0.x, the architecture of the parsing engine has been refactored. At the same

SQL parser facade for openGauss 6.5.2 SQLVisitorFacade SPI Name Description SQLVisitorFacade SQL AST visitor facade Implementation Class Description MySQLS tatementSQLVisitorFacade SQL visitor of statement the following SQL: SELECT id, name FROM t_user WHERE status = 'ACTIVE' AND age > 18 Its parsing AST (Abstract Syntax Tree) is this: 7.2. Sharding 256 Apache ShardingSphere document, v5.1.2 To better adopts ANTLR as a generator for the SQL parsing engine, and uses Visit to obtain SQL Statement from AST. Starting from version 5.0.x, the architecture of the parsing engine has been refactored. At the same

status = 'ACTIVE' AND age > 18 After the above SQL is parsed, its AST (Abstract Syntax Tree) is as follows: The tokens for keywords in the AST are green, while the tokens for variables are red, and gray ones uses Visit to obtain SQL statements from the AST. Since V5.0.x, the architecture of the parsing engine has been restructured and adjusted. Moreover, the AST obtained from the first parsing is stored in ere-sql-parser-mysql ${project.version} • Obtain AST CacheOption cacheOption = new CacheOption(128, 1024L); SQLParserEngine parserEngine = new SQLParserEngine(sql