© XXX Page 1 of 30 Curve元数据节点高可用© XXX Page 2 of 30 1. 需求 2. 技术选型 3. etcd clientv3的concurrency介绍 3.1 etcd clientV3的concurrency模块构成 3.2 Campaign的流程 3.2.1 代码流程说明 3.2.2 举例说明Campagin流程 3.3 Observe的流程 Curve中MDS的选举过程 4.2 图示说明选举流程 4.2.1 正常流程 4.2.2 异常情况1：MDS1退出，可以正常处理 4.2.3 异常情况2：Etcd集群的leader发生重新选举，MDS1未受影响，可以正常处理 4.2.4 异常情况3：Etcd的leader发生重新选举，MDS1受到影响退出，不一定可以正常处理。 4.2.4.1 LeaseTIme < ElectionTime的情况 4 ElectionTime 4.2.4.3 MDS1、MDS2、MDS3的租约全部过期 4.2.4.4 总结 4.2.5 异常情况四: Etcd集群与MDS1(当前leader)出现网络分区 4.2.5.1 事件一先发生 4.2.5.2 事件二先发生 4.2.6 异常情况4：Etcd集群的follower节点异常 4.2.7 各情况汇总 1. 需求 mds是元数据节点，负责空间分配，集群状态监控，集群

the kubernetes API server: export KUBECONFIG=/snap/microk8s/current/client.config Install etcd Install etcd as a StatefulSet into your new Kubernetes cluster. kubectl create -f https://raw.githubusercontent Self-Managed Kubernetes Installa�on using kubeadm Standard Installa�on Installa�on with external etcd Installation using kubeadm Instruc�ons about installing Cilium on Kubernetes cluster deployed by Kubernetes using the cilium-etcd-operator. The cilium-etcd-operator replaces the requirement for an external kvstore. You can learn more about it in the sec�on What is the cilium-etcd-operator? It is suitable

Observing Flows with Hubble Observing flows with Hubble Relay Connectivity Problems Policy Troubleshooting etcd (kvstore) Symptom Library Useful Scripts Reporting a problem Community Weekly Community Meeting Deploy Cilium This is the same helm command as from Install Cilium. However we’re enabling managed etcd and setting both cluster-name and cluster-id for each cluster. Make sure context is set to kind-cluster2 enabled=true \ --set nodePort.enabled=true \ --set hostPort.enabled=true \ --set etcd.enabled=true \ --set etcd.managed=true \ --set identityAllocationMode=kvstore \ --set cluster.name=cluster2

kvstore (etcd). Please refer to the section Installation with external etcd for details on when etcd is required. Quick Installation Installation with managed etcd Installation with external etcd Quick kvstore set up is required which can be set up using an Installation with external etcd or using the Installation with managed etcd. Should you encounter any issues during the installation, please refer to the 75s probe-866bb6f696-tb926 1/1 Running 0 75s Installation with managed etcd The standard Quick Installation guide will set up Cilium to use Kubernetes CRDs to store and propagate

Deploy Cilium This is the same helm command as from Install Cilium. However we’re enabling managed etcd and setting both cluster-name and cluster-id for each cluster. Make sure context is set to kind-cluster2 nodePort.enabled=true \ --set global.hostPort.enabled=true \ --set global.etcd.enabled=true \ --set global.etcd.managed=true \ --set global.identityAllocationMode=kvstore \ --set global nodePort.enabled=true \ --set global.hostPort.enabled=true \ --set global.etcd.enabled=true \ --set global.etcd.managed=true \ --set global.identityAllocationMode=kvstore \ --set global

Observing Flows with Hubble Observing flows with Hubble Relay Connectivity Problems Policy Troubleshooting etcd (kvstore) Symptom Library Useful Scripts Reporting a problem Community Weekly Community Meeting Deploy Cilium This is the same helm command as from Install Cilium. However we’re enabling managed etcd and setting both cluster-name and cluster-id for each cluster. Make sure context is set to kind-cluster2 nodePort.enabled=true \ --set global.hostPort.enabled=true \ --set global.etcd.enabled=true \ --set global.etcd.managed=true \ --set global.identityAllocationMode=kvstore \ --set global

Observing Flows with Hubble Observing flows with Hubble Relay Connectivity Problems Policy Troubleshooting etcd (kvstore) Cluster Mesh Troubleshooting Symptom Library Useful Scripts Reporting a problem Community platforms. For the standard installation path, see Quick Installation. Installation with external etcd Installation on OpenShift OKD Getting Started Using K3s Getting Started Using Kind CNI Chaining External Engine Installation with external etcd This guide walks you through the steps required to set up Cilium on Kubernetes using an external etcd. Use of an external etcd provides better performance and is

Observing Flows with Hubble Observing flows with Hubble Relay Connectivity Problems Policy Troubleshooting etcd (kvstore) Cluster Mesh Troubleshooting Symptom Library Useful Scripts Reporting a problem Community platforms. For the standard installation path, see Quick Installation. Installation with external etcd Installation on OpenShift OKD Getting Started Using K3s Getting Started Using Kind CNI Chaining External Engine Installation with external etcd This guide walks you through the steps required to set up Cilium on Kubernetes using an external etcd. Use of an external etcd provides better performance and is

client的接口层）与S3交互，存取s3 中的对象。 SnapshotDataStore: • SnapshotCloneMetaStore负责管理快照和克隆任务等元数据， 通过调用etcdclient，与etcd存储交互，存取etcd中的快照和克隆 元数据。 SnapshotCloneMetaStore: • CurveClient封装了Client接口，负责与MDS和ChunkServer交互。 CurveClient: 负责管理快照和克隆源卷的引用计数。 SnapshotRef & CloneRef：快照总体流程 • 1.用户发起快照，生成快照任务，并持久化到 etcd，开始执行快照任务。 • 2.在curve中创建内部快照，并返回快照信息， 然后将快照信息更新到etcd。此时，即返回用 户快照成功，可以进行读写。 • 3.向mds查询快照的元数据，转储快照元数据 块metaObject。 • 4.根据快照元数据信息，转储快照数据块 数据 快照流程： chunk chunk chunk chunkserver meta object data object data object S3 Snap Task etcd mds client 3.获取快照元数据 datastore metastore http service clone Task user 快照元数据 2.创建内部快照 5.删除内部快照

1. 1. 1. 2. 1. 2. 3. 4. 3. 2. 背景 将原有的获取chunkid的方法从space迁入mds中，并持久化写入etcd中； 只考虑单 mds 工作的情况； chunkid全局递增。 实现 proto/space.proto 中的 message AllocateS3ChunkRequest、message A KVStorageClient 申请新的chunkid Bundle （etcd中记录的已使用chunkID增加一个chunkID Bundle 的大小）； 申请失败 storeKey_不存在，从现在已使用的 ChunkID 继续申请新的 Bundle 首次连接 etcd etcd 环境切换 storeKey_ 被删除 其他 storeKey_ 在etcd上的Value不可解码 返回一个新的chunkid 返回一个新的chunkid 方案 class ChunkIDGenerator { client_; // etcd chunkIdStoreKey_; // chunkid 存储在etcd上的 key nextId_; // 下一个可分配的 chunkID lastId_; // 当前chunkID bundle 内最后一个可分配的chunkID bundleSize_；// chunkId池子的大小