PFS存储引擎10/17/22 2 Why ●为了减少使用cpu做内存copy，减少系统调用 ●发挥某些被操作系统屏蔽的功能，例如nvme write zero ●根据阿里《When Cloud Storage Meets RDMA》的说法 ●在100Gbps网络带宽时，内存带宽成为瓶颈 ●Intel Memory Latency Checker (MLC)测试得到的CPU内存带宽是 61Gbps10/17/22

a High-performance Software-Defined storage for Cloud Native applications Xiang Dong Architect, Netease Hangzhou Research InstituteAgenda � Why develop storage Design objectives Achievements in Community You can take awayWhy develop Storage requirements ● Hardware requirements ○ more CPU cores, faster Network, nvme storage ● Problems for stateful apps ○ storage capacity expansion ○ capacity imbalance imbalance ○ apps bundled with data locations ● Requirements for elastic block storage ● Requirements for file systemopen-source storage ● Requirements ○ Cloud Native ○ Easy operation and maintenance ○ High

Curve High performance Cloud native Distributed storage system https://www.opencurve.io/Agenda • What is Curve • Use Cases • CurveBS • Key Features • Comparing to Ceph • CurveFS • Comparing Curve • Curve is an distributed storage system • Components • Curve Block Storage (CurveBS) • CurveBS: a high performance cloud native distributed block storage • Curve File System (CurveFS) Database • Data apps(middleware/bigdata/ai) • Data backupContainer • Aggregates underlying storage in the cloud (AWS EBS, AWS S3, AWS Glacier, aliyun EBS, aliyun OSS) or on-prem (baremetal, HDFS

intro • Curve is an distributed storage system • High performance • Easiser operation • Cloud native • Curve is composed of two parts • Curve Block Storage (CurveBS) • CurveBS: a high performance cloud native distributed block storage • Curve File System (CurveFS) • CurveFS: a high performance cloud native file system (Based on CurveBS / S3 compatible Storage)Operator capability Level OPERATOR (by Helm) patch and minor version upgrads supported FULL LIFECYCLE Plan to Support app lifecycle, storage lifecycle(backup, failure, recovery) DEEP INSIGHTS Plan to Support metrics, alerts, log processing

the data in ETCD l Collect cluster status and schedule. 2. Chunkserverl Responsible for data storage l Multi-replicas consistency 3. The client l Provides read and write data interfaces for upper-layer 4. Snapshotcloneserver l Independent of core services l Snapshot data is stored in object storage which supports S3 apis. Therefore, there is no limit of snapshots l Support asynchronous and of chunk can be configured 3. Each file (/foo/bar) contains chunks scattered all over the storage nodes. ChunkServer provides 4KB random read/write capability to support 4KB aligned read/write

High performance Cloud native Distributed storage system https://www.opencurve.io/Agenda • CurveBS Architecture • CurveBS Topology • CurveBS Data Organization • MetaData Server (MDS) • ChunkServer

提高数据可靠性 • 包含多个chunk • 减少复制组数量 类似Ceph中的PG 「Copysets: Reducing the Frequency of Data Loss in Cloud Storage」数据组织形式 • PageFile • 地址空间到—>chunk: 1 : N chunk有先后关系 • 创建时指定大小，lazy分配chunk • 提供4kb随机读写能力数据组织形式

required uint64 len = 4; // file logic length required uint64 size = 5; // file size in object storage }; message S3ChunkInfoList { repeated S3ChunkInfo s3Chunks = 1; }; message UpdateInodeS3VersionRequest

支持块存储、文件存储、对象存储架构简介 — 概念介绍 object：存储单元 PG：Placement Groups 归置组 归置组中的成员为副本 OSD：Object Storage Device, 管理一个磁盘的进程架构简介 — 数据放置 使用多级哈希的方式 使用CRUSH算法根据pgid获得指定的副本个数的id osd.1, osd.2, osd

required uint64 len = 4; // file logic length required uint64 size = 5; // file size in object storage }; message Inode { required uint64 inodeId = 1; required uint32 fsId = 2; required uint64