PFS存储引擎10/17/22 2 Why ●为了减少使用cpu做内存copy，减少系统调用 ●发挥某些被操作系统屏蔽的功能，例如nvme write zero ●根据阿里《When Cloud Storage Meets RDMA》的说法 ●在100Gbps网络带宽时，内存带宽成为瓶颈 ●Intel Memory Latency Checker (MLC)测试得到的CPU内存带宽是 61Gbps10/17/22

a High-performance Software-Defined storage for Cloud Native applications Xiang Dong Architect, Netease Hangzhou Research InstituteAgenda � Why develop storage Design objectives Achievements in Community You can take awayWhy develop Storage requirements ● Hardware requirements ○ more CPU cores, faster Network, nvme storage ● Problems for stateful apps ○ storage capacity expansion ○ capacity imbalance imbalance ○ apps bundled with data locations ● Requirements for elastic block storage ● Requirements for file systemopen-source storage ● Requirements ○ Cloud Native ○ Easy operation and maintenance ○ High

Curve High performance Cloud native Distributed storage system https://www.opencurve.io/Agenda • What is Curve • Use Cases • CurveBS • Key Features • Comparing to Ceph • CurveFS • Comparing Curve • Curve is an distributed storage system • Components • Curve Block Storage (CurveBS) • CurveBS: a high performance cloud native distributed block storage • Curve File System (CurveFS) Database • Data apps(middleware/bigdata/ai) • Data backupContainer • Aggregates underlying storage in the cloud (AWS EBS, AWS S3, AWS Glacier, aliyun EBS, aliyun OSS) or on-prem (baremetal, HDFS

intro • Curve is an distributed storage system • High performance • Easiser operation • Cloud native • Curve is composed of two parts • Curve Block Storage (CurveBS) • CurveBS: a high performance cloud native distributed block storage • Curve File System (CurveFS) • CurveFS: a high performance cloud native file system (Based on CurveBS / S3 compatible Storage)Operator capability Level OPERATOR provisioning and configuration management SEAMLESS UPGRADES Y (by Helm) patch and minor version upgrads supported FULL LIFECYCLE Plan to Support app lifecycle, storage lifecycle(backup, failure, recovery)

the data in ETCD l Collect cluster status and schedule. 2. Chunkserverl Responsible for data storage l Multi-replicas consistency 3. The client l Provides read and write data interfaces for upper-layer 4. Snapshotcloneserver l Independent of core services l Snapshot data is stored in object storage which supports S3 apis. Therefore, there is no limit of snapshots l Support asynchronous and of chunk can be configured 3. Each file (/foo/bar) contains chunks scattered all over the storage nodes. ChunkServer provides 4KB random read/write capability to support 4KB aligned read/write

High performance Cloud native Distributed storage system https://www.opencurve.io/Agenda • CurveBS Architecture • CurveBS Topology • CurveBS Data Organization • MetaData Server (MDS) • ChunkServer

提高数据可靠性 • 包含多个chunk • 减少复制组数量 类似Ceph中的PG 「Copysets: Reducing the Frequency of Data Loss in Cloud Storage」数据组织形式 • PageFile • 地址空间到—>chunk: 1 : N chunk有先后关系 • 创建时指定大小，lazy分配chunk • 提供4kb随机读写能力数据组织形式

required uint64 len = 4; // file logic length required uint64 size = 5; // file size in object storage }; message S3ChunkInfoList { repeated S3ChunkInfo s3Chunks = 1; }; message UpdateInodeS3VersionRequest

支持块存储、文件存储、对象存储架构简介 — 概念介绍 object：存储单元 PG：Placement Groups 归置组 归置组中的成员为副本 OSD：Object Storage Device, 管理一个磁盘的进程架构简介 — 数据放置 使用多级哈希的方式 使用CRUSH算法根据pgid获得指定的副本个数的id osd.1, osd.2, osd

required uint64 len = 4; // file logic length required uint64 size = 5; // file size in object storage }; message Inode { required uint64 inodeId = 1; required uint32 fsId = 2; required uint64