## HBase Read Path h openi nx@apache.org ## Abstract ☐ A i ent Si de ☐ Server Si de Tuni ng ## Part-1 Client Si de ## A i ent Scanner  and shorter STWtime. ## End-to-end off heap on the read-path (HBASE-114)  at ed i sue BASE-21206 • Bl ockSi ze ? ## Part - 3 Tuni ng ## Tuni ng • Read Di st ri but i on • Locality • Short Circuit Read • CacheHitRatio • St or eFi I eCount • ReadRawCells / ResponseCells

Background For reducing the Java GC impact to p99/p999 RPC latency, HBase 2.x has made an off-heap read and write path. The KV are allocated from the JVM off-heap, and the memory area in off-heap won't to manage the off-heap memory. Similarly, on the read path, we'll try to read the BucketCache firstly, if the Cache misses, go to the HFile and read the corresponding block. The workflow: reading block Young GC:  Picture.1 QPS and latencies in 100% Get Case (5 Nodes) ![Image](/uploads/documents/e/b/1/a/eb1a5d49f37599869653c47ad3a89554/p2_2

client_version_min: client_revision: http_method: http_user_agent: quota_key: elapsed: is_cancelled: read_rows: read_bytes: total_rows_approx: written_rows: written_bytes: memory_usage: peak_memory_usage: Float32, pages_distinct UInt32, sl_rate_type String, sl_rate_limit UInt16, hosted_on String, read_only UInt8, replica_lag UInt64, role String ## Queries Causing the Most Load normalized query, 00%)3.37 k QPS145.57 m (100.00%)6.21 ms avg1SELECT sbtest558...5.97 (28.54%)1.53 k QPS

0 码力 | 43 页 | 2.70 MB | 1 年前

3

362 7.6.1 TiFlash_schema_error 363 7.6.2 TiFlash_schema_apply_duration 363 7.6.3 TiFlash_raft_read_index_duration 363 7.6.4 TiFlash_raft_wait_index_duration 364 ## 8 Troubleshoot 8.1 TiDB Troubleshooting AUTO_RANDOM ..... 433 8.7.5 Optimization of small table hotspots ..... 436 8.8 Troubleshoot Increased Read and Write Latency ..... 436 8.8.1 Common causes ..... 436 8.8.2 Other causes ..... 440 8.9 TiDB 658 10.2.2 Architecture 659 10.2.3 Configuration 662 10.3 Read Historical Data 666 10.3.1 Use Stale Read (Recommended) 666 10.3.2 Read Historical Data Using the System Variable $ tidb\_snapshot

.... 175 Troubleshoot ..... 175 Elevated query latency on master ..... 176 Master starts up read-only ..... 176 Vitess sees the wrong tablet as master ..... 176 User Guides ..... 176 Prerequisites cross-cell traffic for both data and metadata. While it may be useful to also have the ability to route read traffic to individual cells, Vitess currently serves reads only from the local cell. Writes will go just like reading from a MySQL database. However, depending on the consistency requirements of the read operation, Vitess might fetch the data from a master database or from a replica. By routing each query

.... 166 Troubleshoot ..... 167 Elevated query latency on master ..... 167 Master starts up read-only ..... 167 Vitess sees the wrong tablet as master ..... 168 User Guides ..... 168 Concepts cross-cell traffic for both data and metadata. While it may be useful to also have the ability to route read traffic to individual cells, Vitess currently serves reads only from the local cell. Writes will go just like reading from a MySQL database. However, depending on the consistency requirements of the read operation, Vitess might fetch the data from a master database or from a replica. By routing each query

378 7.6.1 TiFlash_schema_error 378 7.6.2 TiFlash_schema_apply_duration 378 7.6.3 TiFlash_raft_read_index_duration 378 7.6.4 TiFlash_raft_wait_index_duration 379 8 Troubleshoot 379 8.1 TiDB Troubleshooting using AUTO_RANDOM 449 8.7.5 Optimization of small table hotspots 452 8.8 Troubleshoot Increased Read and Write Latency 452 8.8.1 Common causes 452 8.8.2 Other causes 456 8.9 TiDB Cluster Troubleshooting architecture ..... 690 10.3.2 Configuration ..... 691 10.4 Read Historical Data ..... 698 10.4.1 Use Stale Read (Recommended) ..... 698 10.4.2 Read Historical Data Using the System Variable $ tidb\_snapshot

TiFlash_schema_error ..... 394 7.6.2 TiFlash_schema_apply_duration ..... 394 7.6.3 TiFlash_raft_read_index_duration ..... 395 7.6.4 TiFlash_raft_wait_index_duration ..... 395 7.7 Customize Configurations AUTO_RANDOM.....469 8.7.5 Optimization of small table hotspots.....472 8.8 Troubleshoot Increased Read and Write Latency.....472 8.8.1 Common causes.....472 8.8.2 Other causes.....476 8.9 Use PLAN Deployment architecture 716 10.3.2 Configuration 717 10.4 Read Historical Data 724 10.4.1 Use Stale Read (Recommended) 724 10.4.2 Read Historical Data Using the System Variable $ tidb\_snapshot

. 76 What are good videos to watch to get started learning about Vitess? ..... 76 Where can I read additional FAQs? ..... 76 Get Started ..... 76 Helm Chart (deprecated) ..... 76 Prerequisites query latency on master 306 Elevated query latency on master 306 Master starts up read-only 307 Master starts up read-only 307 Vitess sees the wrong tablet as master 307 Vitess sees the wrong tablet cross-cell traffic for both data and metadata. While it may be useful to also have the ability to route read traffic to individual cells, Vitess currently serves reads only from the local cell. Writes will go

deserialization-cache-size: - "1000" 发送到 API 服务器的客户端请求或 API 调用数量由每秒的 Queries(QPS)值和 API 服务器处理的并发请求数决定。客户端可能会过量 QPS 速率发出的请求数量取决于突发值，这对具有极限的应用程序来说非常有用，并可执行不监管的请求数量。当 API 服务器处理大量并发请求时，对请求的响应时间，特别是大型和/或高密度的集群。建议您监控 /或高密度的集群。建议您监控 Prometheus 中的 apiserver_request_count 速率指标，并相应地调整 maxRequestsInFlight 和 QPS。 更改默认值时，需要有一个很好的平衡，因为 API 服务器的 CPU 和内存消耗，etcd IOPS 会在并行处理更多请求时增加。另请注意，大量非 watch 请求可能会在固定 60 秒超时后取消 API 服务器过载，客户端开始重试。 以上提到的因素并浏览了 maxRequestsInFlight、API qps 和 burst 值 *_/etc/origin/master/master-config.yaml masterClients: openshiftLoopbackClientConnectionOverrides: burst: 600 qps: 300 servingInfo: maxRequestsInFlight: