© 2018 Percona. 1 Peter Zaitsev Analyzing MySQL Logs with ClickHouse CEO, Percona April 27,2018 © 2018 Percona. 2 ClickHouse is my love at the first sight © 2018 Percona. 3 Why ? Fast and 5 Why Keep and Analyze Raw Logs ? Raw Logs Contain ultimate level of detail You build any summaries from Raw Events You can’t recover Raw Events from Summaries Audit Trace Application/User Path Path © 2018 Percona. 6 Why Not ? Generating Full Query Logs can be Expensive Logs can Consume a lot of Space Logs can be expensive to query © 2018 Percona. 7 Clickhouse Answers • 10x+

PRESENTS Vitess security audit In collaboration with the Vitess maintainers, Open Source Technology Improvement Fund and The Linux Foundation Authors Adam Korczynski David Korczynski Commons 4.0 (CC BY 4.0) Vitess Security Audit, 2023 Table of contents Table of contents 1 Executive summary 2 Notable findings 3 Project Summary 4 Audit Scope 4 Threat model formalisation 5 Fuzzing Conclusions 40 1 Vitess Security Audit, 2023 Executive summary In March and April 2023, Ada Logics carried out a security audit of Vitess. The primary focus of the audit was a new component of Vitess,

TiDB Audit Plugin User Guide August 4, 2022 TiDB Audit Plugin User Guide Introduction The TiDB audit plugin records the TiDB server’s activities that are expected to follow auditing regulations of describes how to compile, package, and use the audit plugin. Download the plugin You can download the plugin on TiDB Enterprise Edition Downloads. Deploy the audit plugin After downloading the plugin, you TiUP to deploy the audit plugin. Use TiDB Operator to deploy the plugin Configure TidbCluster CR. tidb: additionalContainers: - command: - sh - -c - touch /var/log/tidb/tidb-audit.log; tail -n0 -F

database auditing and the audit plugin 1 Obtain the database auditing feature 2 The range of database auditing 2 The events of database auditing 3 Recorded information in audit logs 5 General information 6 Audit operation information 7 Audit log filters and rules 7 Filters 8 Filter rules 9 File formats of audit log 10 Rotation of audit log 11 The number and duration for reserving audit logs 11 Audit log redaction 11 System tables 11 mysql.audit_log_filters 11 mysql.audit_log_filter_rules 12 System variables 13 tidb_audit_enabled 13 tidb_audit_log 14 tidb_audit_log_format 14 tidb_au

Translator . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 417 Sharding Audit Algorithm . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 418 Data Masking Algorithm Implementation classes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 453 10.5 SQL Audit . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 453 10.5.1 auditStrategy: # Sharding audit strategy auditorNames: # Sharding auditor name - - allowHintDisable: true # Enable or disable sharding audit hint autoTables: # Auto Sharding

Translator . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 448 Sharding Audit Algorithm . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 448 Data Masking Algorithm Implementation classes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 482 10.5 SQL Audit . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 482 10.5.1 auditStrategy: # Sharding audit strategy auditorNames: # Sharding auditor name - - allowHintDisable: true # Enable or disable sharding audit hint autoTables: # Auto Sharding

· · · · · · · · · · · · · · · · · · · · · · · · · · · · · · · · · · · 380 6.6.2 TiFlash critical logs · · · · · · · · · · · · · · · · · · · · · · · · · · · · · · · · · · · · · · · · · · · · 380 6.6.3 protection mechanism to avoid panic caused by disk space drainage #10537 – Support archiving and rotating logs #11651 – Reduce the system call by the Raft client and increase CPU efficiency #11309 – Coprocessor BR performs encrypted backup #29794 – TiCDC * Reduce the count of “EventFeed retry rate limited” logs #4006 * Reduce the replication latency when replicating many tables #3900 * Reduce the time for the

Translator . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 310 Sharding Audit Algorithm . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 311 4.4 Error Code . . . . auditStrategy: # Sharding audit strategy auditorNames: # Sharding auditor name - - allowHintDisable: true # Enable or disable sharding audit hint autoTables: # Auto Sharding 1 # Sharding audit algorithm configuration auditors: audit-algorithm-name> (+): # Sharding audit algorithm name type: # Sharding audit algorithm type props: # Sharding audit algorithm properties

· · · · · · · · · · · · · · · · · · · · · · · · · · · · · · · · · · · 804 8.6.2 TiFlash critical logs · · · · · · · · · · · · · · · · · · · · · · · · · · · · · · · · · · · · · · · · · · · · 804 8.6.3 default log storage engine Since v6.1.0, TiDB uses Raft Engine as the default storage engine for logs. Compared with RocksDB, Raft Engine can reduce TiKV I/O write traffic by up to 40% and CPU usage by 3.1 or v5.4.0 to v6.0.0 or later versions #12269 • Fix the issue of OOM caused by appending Raft logs when the memory resource is insufficient #11379 • Fix the issue of TiKV panic caused by the race

· · · · · · · · · · · · · · · · · · · · · · · · · · · · · · · · · · · 986 8.8.2 TiFlash critical logs · · · · · · · · · · · · · · · · · · · · · · · · · · · · · · · · · · · · · · · · · · · · 986 8.8.3 .pingcap.com/files/2023/09/18204824/TiDB-Database �→ -Auditing-User-Guide1.pdf" target="_blank">Audit log enhancement �→ ( �→ Enterprise user-friendly filter �→ settings, a new file output format in JSON, and lifecycle management �→ of audit logs. 2.2.1 Feature details 2.2.1.1 Performance • Enhance the Partitioned Raft KV storage engine