HBase Read Path openinx@apache.org Abstract ❏ Client Side ❏ Server Side ❏ Tuning Part-1 Client Side HBase Client ClientScanner ClientScanner cache(queue) scanner.next() RegionServer-0 RegionServer-1 (old generation) ● Less mixed GC(s) and shorter STW time. End-to-end offheap on the read-path (HBASE-11425) BucketCache StoreFileScanner Copy the Block from BucketCache(offheap) to onheap. Rpc Handler MaxResultSize ● Timeout ○ HeartBeat: abort this rpc once timeout and just return the current results to client. ○ Cursor: return a fake result with the current rowkey for next rpc once timeout. ● Batch ○

HBase Practice At Xiaomi huzheng@xiaomi.com About This Talk ● Async HBase Client ○ Why Async HBase Client ○ Implementation ○ Performance ● How do we tuning G1GC for HBase ○ CMS vs G1 ○ Tuning Tuning G1GC ○ G1GC in XiaoMi HBase Cluster Part-1 Async HBase Client Why Async HBase Client ? Request-1 Response-1 Request-2 Response-2 Request-3 Response-3 Request-4 Response-4 Request-1 RPC-3 RPC-4 RPC-1 RPC-2 RPC-3 RPC-4 Blocking Client (Single Thread) Non-Blocking Client(Single Thread) Fault Amplification When Using Blocking Client RegionServer RegionServer RegionServer Handler-1

HBase Practice At XiaoMi tianjy1990@gmail.com openinx@apache.org Part-1 Problems In Practice Problems in XiaoMi ❏ Problem 1. How to satisfy the regular demand of scanning table without affecting analysis need to scan a large number of data from hbase ❏ They are executed by mapreduce or spark, that put a heavy burden on HBase Scan snapshot directly ❏ HBase already provides this feature: TableSnapshotInputFormat TableSnapshotInputFormat (ClientSideRegionScanner) ❏ Construct regions by snapshot files ❏ Read data without any HBase RPC requests ❏ Required READ access to reference files and HFiles Snapshot ACL ❏ HDFS ACL could

… … Stack Traces for Async Code are Unhelpful Threadpool threads: #0 process_file(...) #5 pool.run() #10 __clone … … IO Thread: #0 async_read_some_at(...) #3 ctx Threadpool threads: #0 process_file(...) #5 pool.run() #10 … IO Thread: … #0 async_read_some_at(...) #3 ctx.run() #5 __clone … … Main thread: #0 unifex::sync_wait( … #0 async_read_some_at(...) … #12 unifex::sync_wait(...) #16 main() #19 __libc_start_main() … … Async Stacks are BetterStructured Concurrency Makes Async Stacks Possible

Node.js* Client & Web Bridge Ready for ROS* 2.0 Minggang Wang September 30 2018 Agenda ● Who We Are ● Why Use Node.js & Web in Robotics ● Thinking in “ROS 2.0 + Web” ● What We Have Done for “ROS of Intel's open source development efforts. ● We’re from OTC Web Team; we do web technology in client, edge, cloud, IoT, W3C standard, Robotics & etc., to keep web open, secure, rich-featured and performant 2 packages. Both hosted in GitHub RWT thanks to Jihoon ● rclnodejs (github repo) It’s a Node.js client of ROS 2.0. It provides fast, easy & powerful JavaScript API of ROS 2.0 ● ros2-web-bridge (github

HBASE-21879 Read HFile ’s Block into ByteBuffer directly. 1. Background For reducing the Java GC impact to p99/p999 RPC latency, HBase 2.x has made an offheap read and write path. The KV are allocated deallocate the memory explicitly by themself. On the write path, the request packet received from client will be allocated offheap and retained until those key values are successfully written to the WAL HFile and read the corresponding block. The workflow: reading block from cache OR sending cells to client, is basically not involved in heap memory allocations. However, in our performance test at

are: Hive Thrift Protocol A HiveServer2-compatible interface that allows end users to use a thrift client(cross-language support, both tcp and http), a Java Database Connectivity(JDBC) interface over thrift networking between clients and servers. On the data plane, the Kyuubi engines use the same trusted client identities to instantiate themselves. The resource acquirement and data and metadata access all happen Business Intelligence Tools and SQL IDEs ODBC Drivers Thrift APIs RESTful APIs and Clients Web UI Python Client Commons Extension Guide Extensions Server Side Extensions Engine Side Extensions Connectors

are: Hive Thrift Protocol A HiveServer2-compatible interface that allows end users to use a thrift client(cross-language support, both tcp and http), a Java Database Connectivity(JDBC) interface over thrift networking between clients and servers. On the data plane, the Kyuubi engines use the same trusted client identities to instantiate themselves. The resource acquirement and data and metadata access all happen Business Intelligence Tools and SQL IDEs ODBC Drivers Thrift APIs RESTful APIs and Clients Web UI Python Client Commons Extension Guide Extensions Server Side Extensions Engine Side Extensions Connectors

are: Hive Thrift Protocol A HiveServer2-compatible interface that allows end users to use a thrift client(cross-language support, both tcp and http), a Java Database Connectivity(JDBC) interface over thrift networking between clients and servers. On the data plane, the Kyuubi engines use the same trusted client identities to instantiate themselves. The resource acquirement and data and metadata access all happen Business Intelligence Tools and SQL IDEs ODBC Drivers Thrift APIs RESTful APIs and Clients Web UI Python Client Commons Extension Guide Extensions Server Side Extensions Engine Side Extensions Connectors

are: Hive Thrift Protocol A HiveServer2-compatible interface that allows end users to use a thrift client(cross-language support, both tcp and http), a Java Database Connectivity(JDBC) interface over thrift networking between clients and servers. On the data plane, the Kyuubi engines use the same trusted client identities to instantiate themselves. The resource acquirement and data and metadata access all happen Business Intelligence Tools and SQL IDEs ODBC Drivers Thrift APIs RESTful APIs and Clients Web UI Python Client Commons Extension Guide Extensions Server Side Extensions Engine Side Extensions Connectors