HBASE-21879 Read HFile ’s Block into ByteBuffer directly.this is easier to manage the offheap memory. Similarly, on the read path, we’ll try to read the BucketCache firstly, if the Cache misses, go to the HFile and read the corresponding block. The workflow: HFile is still copied to the heap firstly, the heap block won’t free unless the WriterThread of BucketCache flushes the Block to offheap IOEngine successfully. In the pressure test, due to the large amount 2. Cache the HFileBlock in BucketCache which means putting the ByteBuff into RAMCache; 3. WriteThread in BucketCache will flush the ByteBuff into IOEngine in BucketCache asynchronously (don’t block0 码力 | 18 页 | 1.14 MB | 1 年前3
HBase Read PathStoreFileScanner StoreFileScanner …. Revisit Concepts BloomFilter MSLAB scan.setTimeRange BucketCache scan.setBatch scan.setMaxVersion scan.setRaw scan.setStartRow scan.setStopRow scan.setLimit setNeedCursorResult scan.setStopRow scan.setStartRow Filter/FilterList scan.setTimeRange scan.setMaxVersion BucketCache MSLAB BloomFilter Filter/FilterList scan.setRaw Column Qualifer Family TTL scan.setReversed GC (G1) After GC (G1) Memory allocated on Java heap Free memory on Java heap BucketCache Bucket-1(2MB) BucketCache 4KB 4KB 4KB 4KB ….. Bucket-2(2MB) 8KB 8KB 8KB 8KB ….. Bucket-3(2MB)0 码力 | 38 页 | 970.76 KB | 1 年前3
HBase Practice At Xiaomi<= 0.45 ○ hfile.block.cache.size = 0.1 ○ hbase.hregion.memstore.flush.size = 256 MB ○ hbase.bucketcache.ioengine = offheap Test Environment RegionServer RegionServer RegionServer RegionServer RegionServer interval >= 1 sec(Better) Conclusion ● Initial IHOP ○ BucketCache OFF-Heap ■ IHOP > MaxMemstore%Heap + MaxL1CacheSize%Heap + Delta ○ BucketCache On-Heap ■ IHOP > MaxMemstore%Heap + MaxL1CacheSize%Heap0 码力 | 45 页 | 1.32 MB | 1 年前3
共 3 条
- 1













