#10980 – Fix the issue of poor scan performance because MVCC Deletion versions are not dropped by compaction filter GC #11248 • PD – Fix the issue that PD incorrectly delete the peers with data and in pending RocksDB per sec- ond gc. �→ enable �→ - �→ compaction �→ - �→ filter �→ Whether to en- able com- paction filter 319 Configuration item Description gc. �→ compaction �→ - �→ filter �→ - �→ skip �→ - �→ version open �→ - �→ files �→ The total num- ber of files that RocksDB can open {db- �→ name �→ }. �→ compaction �→ - �→ readahead �→ - �→ size �→ The size of readahead �→ dur- ing com- paction {db- �→ name

file storage. �→ flow- �→ control �→ .soft- �→ pending �→ - �→ compaction �→ -bytes- �→ limit Newly added When the pending compaction bytes in KvDB reach this threshold, the flow control mechanism starts file storage. �→ flow- �→ control �→ .hard- �→ pending �→ - �→ compaction �→ -bytes- �→ limit Newly added When the pending compaction bytes in KvDB reach this threshold, the flow control mechanism rejects reduces the impact on the stability of foreground write. In specific, when the stress of RocksDB compaction accumulates, flow control is per- formed at the TiKV scheduler layer instead of the RocksDB layer

Description TiKV con- figuration file soft- �→ pending �→ - �→ compaction �→ -bytes- �→ limit Modified The soft limit on the pending compaction bytes. The default value is changed from "64GB" to "192GB". requests, TiKV Write Rate Limiter smoothes the write traffic of TiKV background tasks such as GC and Compaction. The default value of TiKV background task write rate limiter is “0MB”. It is recommended to set results when cloning shared delta index concurrently – Fix the potential panic that occurs when the Compaction Filter feature is enabled – Fix the issue that TiFlash cannot resolve the lock fallen back from

Fix the issue that GC scan causes memory overflow #11410 – Fix the issue that RocksDB flush or compaction causes panic when the disk ca- pacity is full #11224 • PD – Fix the issue that Region statistics writ- ten into RocksDB per sec- ond gc. �→ enable �→ - �→ compaction �→ - �→ filter �→ Whether to en- able com- paction filter gc. �→ compaction �→ - �→ filter �→ - �→ skip �→ - �→ version �→ - �→ check ber of files that RocksDB can open 411 Configuration item Description {db- �→ name �→ }. �→ compaction �→ - �→ readahead �→ - �→ size �→ The size of readahead �→ dur- ing com- paction {db- �→ name

information, see documentation. • Support periodic full compaction (experimental) #12729 afeinberg Starting from v7.6.0, TiDB supports periodic full compaction for TiKV. This feature serves as an enhancement feature to perform data compaction during idle periods to improve the performance during peak periods. You can set the specific times that TiKV initiates periodic full compaction by configur- ing the TiKV periodic full compaction by configuring periodic �→ -full-compact-start-max-cpu. The default value of periodic-full-compact- �→ start-max-cpu is 10%, which means that periodic full compaction is triggered

Support non-transactional DML statements (only support DELETE) • TiFlash supports on-demand data compaction • MPP introduces the window function framework • TiCDC supports replicating changelogs to Kafka statement, which provides a manual way to compact physical data based on the existing background compaction mechanism. With this statement, you can update data in earlier formats and improve read/write performance writ- ten into RocksDB per sec- ond gc. �→ enable �→ - �→ compaction �→ - �→ filter �→ Whether to en- able com- paction filter gc. �→ compaction �→ - �→ filter �→ - �→ skip �→ - �→ version �→ - �→ check

accidentally delete valid data #17258 @hbisheng • Fix the issue that Ingestion picked level and Compaction Job Size(files) are displayed incorrectly in the TiKV dashboard in Grafana #15990 @Connor1996 Note that it is recommended to reserve 20% of storage space, because background tasks such as compaction and snapshot replication also consume a portion of the storage space. 6.3.4 Change configuration configurable. On key rotation, TiKV does not rewrite all existing files to replace the key, but RocksDB compaction are expected to rewrite old data into new data files, with the most recent data key, if the cluster

to spill-dir, ensuring continuous operation of the system #17356 @LykxSassinator • Optimize the compaction trigger mechanism of RocksDB to accelerate disk space reclamation when handling a large number Note that it is recommended to reserve 20% of storage space, because background tasks such as compaction and snapshot replication also consume a portion of the storage space. 6.3.4 Change configuration regularly update the metadata. Because the time interval between the MVCC of etcd and PD’s default compaction is one hour, the amount of PD storage that TiCDC uses is proportional to the amount of metadata

to spill-dir, ensuring continuous operation of the system #17356 @LykxSassinator • Optimize the compaction trigger mechanism of RocksDB to accelerate disk space reclamation when handling a large number Note that it is recommended to reserve 20% of storage space, because background tasks such as compaction and snapshot replication also consume a portion of the storage space. 6.3.4 Change configuration configurable. On key rotation, TiKV does not rewrite all existing files to replace the key, but RocksDB compaction are expected to rewrite old data into new data files, with the most recent data key, if the cluster

Note that it is recommended to reserve 20% of storage space, because background tasks such as compaction and snapshot replication also consume a portion of the storage space. 6.3.4 Change configuration V2 DTFiles and will gradually rewrite existing V2 DTFiles to V3 DTFiles during subsequent data compaction. After upgrading TiFlash to v7.3 and configuring TiFlash to use V3 DTFiles, if you need to revert from v7.4, to reduce the read and write amplification generated during data compaction, TiFlash optimizes the data compaction logic of PageStorage V3, which leads to changes to some of the underlying storage