作 • RaftService。Braft内置的service， 完成raft成员之间的选举，日志复制， 安装快照等操作。 ChunkServer架构CopysetNode封装了braft的Node，并 实现了braft的状态机，完成与raft的交 互。详细交互流程后面展开。 CopysetNodeManager负责管理 CopysetNode的创建、初始化、删除等 ChunkServer架构心跳模块有两方面的职责： 心跳上报时携带这些信息。ChunkServer核心模块-CopysetNode 写请求： 1. Client发送写请求给Leader ChunkServer 2. 请求封装，提交给Raft node 3. 本地持久化entry的同时发送给其他peer 4. 本地持久化log entry成功，并且有一个peer也落 盘成功，则commit 5. Commit后apply，此时把写请求写 from CS1 to CS2的operator给这三个copyset ④ MDS通过RPC在CS2上创建copyset1,2,3这三个copyset ⑤ 假定三个copyset的leader都是CS3，在CS3的下一次心跳的 response中，下发第三步生成的三个operator ⑥ CS3收到change peer from CS1 to CS2的operator，给CS2同步 raft

service (in Plan) • Config CurveBS by (Cluster and Pool CRDs) in Kubernetes (in Plan) • Support Operator capability level 5 (in Plan) • horizontal / vertical scaling, auto config tuning, abnormal detection (RAFT) CEPH WRITE SUCCESS majority write successful all write successful READ Leader of copyset Node in PG SLOW STORAGE/DISK FAILURE INFLUENCE without I/O disruption I/O jitter occasionally CAN SYNC SYNC WITH REMOTE DISK SERVER Y NI/O Jitter (vs. Ceph) 3 replicas with 9 nodes cluster each node has 20 x SSD, 2xE5-2660 v4 and 256GB mem FAULTS CASE CURVE I/O JITTER CEPH I/O JITTER COMMENT ONE DISK

high performance cloud native file system (Based on CurveBS / S3 compatible Storage)Operator capability Level OPERATOR CAPABILITY LEVEL CURVE COMMENT BASIC INSTALL Y (by Helm) automated application provisioning horizontal/vertical scaling, auto config tuning, abnormal detection, schedule tuningCloud native Feature list • Features for Cluster • Features for CurveBS • Features for CurveFS • Features for monitorFeature

services can be configured using CRDs in public cloud and on- premises environments • Use the Curve operator to install, upgrade, backup, and expand CurveBS clusters • Use Curve Discover to discover resource services can be configured using CRDs in public cloud and on- premises environments • Use the Curve operator to install, upgrade, backup, and expand CurveFS clusters • Use Curve Discover to discover resource

改造 vs 自己实现? redis 中哈希表实现的优点？ 参考 前言 根据之前讨论的结果，元数据节点的架构如下图所示，这里涉及到两部分需要持久化/编码的内容： Raft Log：记录 operator log Raft Snapshot：将内存中的数据结构以特定格式 dump 到文件进行持久化© XXX Page 3 of 12 Raft Log +------+-----------

architecture, data organization and topology structure of CURVE. CurveBS uses the central node Metadata Server (MDS) to manage virtual disk mapping and data replicas distribution. Decentralization in CurveBS This section mainly describes how data nodes manage chunk. Each node in the figure represents a data node (ChunkServer). A CopySet is the basic unit of data replication in CurveBS. A complete, performance recovers. 3. The Client queries the ChunkServer for the leader ChunkServer node of the copyset where chunk resides. 4. The Client sends read/write requests to the leader (IP=127

on管理剩下的一直到2^63-1的Inode id。创建meta partition的时候，选择的3个meta node组成一个复制组。如何选择？论文上写的是按照存储节点的memory和disk usage来选的，通常选择内存和disk使用率最低的节点。 并去对应的meta node上去创建对应的meta partition。 如何选择partition的host，通过这个函数去选择。 func

3.html https://juejin.cn/post/6844903923048792078 https://www.gnu.org/software/libc/manual/html_node/File-Status-Flags.html https://android.googlesource.com/platform/prebuilts/gcc/linux-x86/host/x86_64-linux-glibc2 6/+/refs/heads/jb-dev/sysroot/usr/include/asm-generic/fcntl.h https://www.gnu.org/software/libc/manual/html_node/Permission-Bits.html https://xinqiu.gitbooks.io/linux-insides-cn/content/SysCall/linux-syscall-5

●当前取舍的：TCP总是可用的， UCX作为选项38 Ucp Context ●只有一个全局对象，使用下列函数获取 ●UCP_Context* get_or_create_ucp_ctx() ●指定了FEATURE_AM, 多线程共享39 命令行参数控制context的属性 ●--brpc_ucp_error_mode缺省是none，是的本地通讯使用shared memory成为可能 ●--brpc_set_cpu_latency

26%Performance compare with CEPH The test environment ● A cluster consists of six nodes. Each node consists of 20 x SSDS, 256GB memory, and 2 x e5-2660 cpus Performance using RDMA ● Compared with