PFS存储引擎10/17/22 2 Why ●为了减少使用cpu做内存copy，减少系统调用 ●发挥某些被操作系统屏蔽的功能，例如nvme write zero ●根据阿里《When Cloud Storage Meets RDMA》的说法 ●在100Gbps网络带宽时，内存带宽成为瓶颈 ●Intel Memory Latency Checker (MLC)测试得到的CPU内存带宽是 61Gbps10/17/22 const struct iovec *iov, int iovcnt, off_t offset); ●IO vector的接口主要是为了与brpc的iobuf对接，iobuf由若干地址不连 续的block组成，一次IO提交可以提高效率。10/17/22 8 PFS+SPDK 的部分读写的实现 ●某些盘只支持4k单位读写，但是CurveBS支持512字节读写 ●可能存在部分写的并发冲突 ●引入并发的range

integrate to Cloud NativeCurve intro • Curve is an distributed storage system • High performance • Easiser operation • Cloud native • Curve is composed of two parts • Curve Block Storage (CurveBS) high performance cloud native distributed block storage • Curve File System (CurveFS) • CurveFS: a high performance cloud native file system (Based on CurveBS / S3 compatible Storage)Operator capability (by Helm) patch and minor version upgrads supported FULL LIFECYCLE Plan to Support app lifecycle, storage lifecycle(backup, failure, recovery) DEEP INSIGHTS Plan to Support metrics, alerts, log processing

a High-performance Software-Defined storage for Cloud Native applications Xiang Dong Architect, Netease Hangzhou Research InstituteAgenda � Why develop storage Design objectives Achievements in Community You can take awayWhy develop Storage requirements ● Hardware requirements ○ more CPU cores, faster Network, nvme storage ● Problems for stateful apps ○ storage capacity expansion ○ capacity imbalance bundled with data locations ● Requirements for elastic block storage ● Requirements for file systemopen-source storage ● Requirements ○ Cloud Native ○ Easy operation and maintenance ○ High performance

Curve High performance Cloud native Distributed storage system https://www.opencurve.io/Agenda • What is Curve • Use Cases • CurveBS • Key Features • Comparing to Ceph • CurveFS • Comparing Curve • Curve is an distributed storage system • Components • Curve Block Storage (CurveBS) • CurveBS: a high performance cloud native distributed block storage • Curve File System (CurveFS) high performance cloud native file systemUse Cases • Container • Database • Data apps(middleware/bigdata/ai) • Data backupContainer • Aggregates underlying storage in the cloud (AWS EBS, AWS

High performance Cloud native Distributed storage system https://www.opencurve.io/Agenda • CurveBS Architecture • CurveBS Topology • CurveBS Data Organization • MetaData Server (MDS) • ChunkServer ChunkServer • Client • CurveBS IO processing flow • CurveBS Performance considerations • Cloud NativeAgenda • CurveFS Architecture • CurveFS Data Organization • CurveFS file Organization • polardb for postgresql using CurveBS • Cloud-native support for CurveBS • CurveBS clusters and related monitoring services can be configured using CRDs in public cloud and on- premises environments •

the data in ETCD l Collect cluster status and schedule. 2. Chunkserverl Responsible for data storage l Multi-replicas consistency 3. The client l Provides read and write data interfaces for upper-layer 4. Snapshotcloneserver l Independent of core services l Snapshot data is stored in object storage which supports S3 apis. Therefore, there is no limit of snapshots l Support asynchronous and virtual block device in CurveBS1. CurveBS maps the virtual block device to a file. For example, block device /dev/sda corresponds to file /foo/bar in CurveBS 2. The address space of the block device

提高数据可靠性 • 包含多个chunk • 减少复制组数量 类似Ceph中的PG 「Copysets: Reducing the Frequency of Data Loss in Cloud Storage」数据组织形式 • PageFile • 地址空间到—>chunk: 1 : N chunk有先后关系 • 创建时指定大小，lazy分配chunk • 提供4kb随机读写能力数据组织形式

可以理解为一组复制组，这组复制组的成员关系完全一样。CopySet的概念在文献「Copysets: Reducing the Frequency of Data Loss in Cloud Storage」提出。 在 Curve 系统引入 CopySet 有几个目的： 1. 减少元数据量：如果为每个Chunk去保存复制组成员关系，需要至少 ChunkID+3×NodeID=20 个byte，而如

metaserver.proto space相关数据结构和proto 关键流程 init流程 write流程 read流程 整体架构 S3ClientAdaptor模块：负责将文件数据进行chunk，以及block的拆分为s3的object，并写入/读取s3的object。 S3-allocator模块：负责分配s3-object唯一标识。© XXX Page 3 of 11 整体思路 s3不需要释放空间，可 直接删除对应s3 object） 文件首先会按照chunk进行拆分，每个chunk固定64M/1G（待定），chunk内部会划分为多个block，每个block最大4M，每个block对应s3上一个object。 s3上对象已chunkid_indexblock_version进行命名，元数据则已S3ChunkInfo（见数据结构）的方式存储 required uint64 len = 4; // file logic length required uint64 size = 5; // file size in object storage }; message S3ChunkInfoList { repeated S3ChunkInfo s3Chunks = 1; }; message UpdateInodeS3VersionRequest

raft在Curve中的应用 05 Q&A 04 Curve对raft的优化Curve块存储RAFT应用 Curve块存储 • 高性能、更稳定、易运维 • 支持NBD(network block device)、iscsi • 支持RDMA和SPDK Curve块存储架构 • client：接受用户请求。 • mds：保存元数据，包括topo信息、块设备信息、 数据分布信息等，持久化到etcd中。 Github：https://github.com/opencurve/curve • 公众号：OpenCurve • 用户群：添加微信号OpenCurve_bot可邀请加群 • Slack：workspace cloud-native.slack.com，channel #project_curve