Handle Edge Cloud Network with KubeBus Yulin Sun, yulin.sun@huawei.com Seattle Cloud Lab, Huawei R&D USA, Bellevue WA Agenda • Edge sample user scenarios • Edge network characteristics • Related work Sample Scenarios HiLens Campus surveillance Huawei Hilens Edge network characteristics • Edge Nodes running at private network • Connect to Cloud behind NAT gateway • Mightn’t have direct connection Cluster Management • There is cloud cluster, edge cluster, i.e. multiple nodes running in private network • Edge nodes, Edge cluster and cloud cluster needs acting as a single cluster Edge Node Management

Handle Edge Cloud Network with KubeBus Yulin Sun, yulin.sun@huawei.com Seattle Cloud Lab, Huawei R&D USA, Bellevue WA Agenda • Edge sample user scenarios • Edge network characteristics • Related work Sample Scenarios HiLens Campus surveillance Huawei Hilens Edge network characteristics • Edge Nodes running at private network • Connect to Cloud behind NAT gateway • Mightn’t have direct connection Cluster Management • There is cloud cluster, edge cluster, i.e. multiple nodes running in private network • Edge nodes, Edge cluster and cloud cluster needs acting as a single cluster Edge Node Management

security policy document may be freely reproduced and distributed in its entirety without modification. Rancher Kubernetes Cryptographic Library FIPS 140-2 Non-Proprietary Security Policy Document Center Rd., Ste. 460 Herndon, VA 20171 corsec.com +1 703.276.6050 FIPS 140-2 Security Policy Rancher Kubernetes Cryptographic Library Page 2 of 16 References Ref Full Specification FIPS 198-1, The Keyed Hash Message Authentication Code (HMAC) 7/16/2008 FIPS 140-2 Security Policy Rancher Kubernetes Cryptographic Library Page 3 of 16 Acronyms and Definitions Term

OPERATOR 概述 概述 3.1. CLUSTER NETWORK OPERATOR 3.2. DNS OPERATOR 3.3. INGRESS OPERATOR 第 第 4 章 章 OPENSHIFT CONTAINER PLATFORM 中的 中的 CLUSTER NETWORK OPERATOR 4.1. CLUSTER NETWORK OPERATOR 4.2. 查看集群网络配置 4 4.3. 查看 CLUSTER NETWORK OPERATOR 状态 4.4. 查看 CLUSTER NETWORK OPERATOR 日志 4.5. CLUSTER NETWORK OPERATOR 配置 4.6. 其他资源 第 第 5 章 章 OPENSHIFT CONTAINER PLATFORM 中的 中的 DNS OPERATOR 5.1. DNS OPERATOR 5.2. 更改 VRF 分配从属网络 第 第 14 章 章 硬件网 硬件网络 络 14.1. 关于单根 I/O 虚拟化（SR-IOV）硬件网络 14.2. 安装 SR-IOV NETWORK OPERATOR 14.3. 配置 SR-IOV NETWORK OPERATOR 14.4. 配置 SR-IOV 网络设备 14.5. 配置 SR-IOV 以太网网络附加 72 73 76 76 78 79 79

OPERATOR 概述 概述 3.1. CLUSTER NETWORK OPERATOR 3.2. DNS OPERATOR 3.3. INGRESS OPERATOR 第 第 4 章 章 OPENSHIFT CONTAINER PLATFORM 中的 中的 CLUSTER NETWORK OPERATOR 4.1. CLUSTER NETWORK OPERATOR 4.2. 查看集群网络配置 4 4.3. 查看 CLUSTER NETWORK OPERATOR 状态 4.4. 查看 CLUSTER NETWORK OPERATOR 日志 4.5. CLUSTER NETWORK OPERATOR 配置 4.5.1. Cluster Network Operator 配置对象 defaultNetwork 对象配置 配置 OpenShift SDN CNI 集群网络供应商 配置 OVN-Kubernetes OVN-Kubernetes CNI 集群网络供应商 kubeProxyConfig 对象配置 4.5.2. Cluster Network Operator 配置示例 4.6. 其他资源 第 第 5 章 章 OPENSHIFT CONTAINER PLATFORM 中的 中的 DNS OPERATOR 5.1. DNS OPERATOR 5.2. 查看默认 DNS 5.3. 使用 DNS 转发

网 网络 络 OPERATOR 概述 概述 4.1. CLUSTER NETWORK OPERATOR 4.2. DNS OPERATOR 4.3. INGRESS OPERATOR 4.4. 外部 DNS OPERATOR 4.5. INGRESS NODE FIREWALL OPERATOR 4.6. NETWORK OBSERVABILITY OPERATOR 第 第 5 章 章 OPENSHIFT PLATFORM 中的 中的 CLUSTER NETWORK OPERATOR 5.1. CLUSTER NETWORK OPERATOR 5.2. 查看集群网络配置 5.3. 查看 CLUSTER NETWORK OPERATOR 状态 5.4. 查看 CLUSTER NETWORK OPERATOR 日志 5.5. CLUSTER NETWORK OPERATOR 配置 5.6. 其他资源 VRF 分配从属网络 第 第 26 章 章 硬件网 硬件网络 络 26.1. 关于单根 I/O 虚拟化（SR-IOV）硬件网络 26.2. 安装 SR-IOV NETWORK OPERATOR 26.3. 配置 SR-IOV NETWORK OPERATOR 26.4. 配置 SR-IOV 网络设备 26.5. 配置 SR-IOV 以太网网络附加 26.6. 配置 SR-IOV INFINIBAND

集群上监控的一个关键指标是每个 etcd 集群成员上的 etcd 网络对延迟 的 p99 百分比。使用 Prometheus 跟踪指标数据。 histogram_quantile（ （0.99, rate(etcd_network_peer_round_trip_time_seconds_bucket[2m]) 指标报 告 etcd 在成员间复制客户端请求的时间。确保它小于 50 ms。 其他 其他资 资源 源 $ data:text/plain;charset=US- ASCII,%23%20turn%20on%20Receive%20Flow%20Steering%20%28RFS%29%20for%20all %20network%20interfaces%0ASUBSYSTEM%3D%3D%22net%22%2C%20ACTION%3D% 3D%22add%22%2C%20RUN%7Bprogram%7D%2B%3D%22/bin/bash%20- 程规 规格示例 格示例 $ oc delete -f 05-worker-kernelarg-hpav.yaml network="net01"/> ... <

Configure Kernel Runtime Parameters Configure etcd user and group Ensure that all Namespaces have Network Policies defined Reference Hardened RKE cluster.yml configuration Reference Hardened RKE Template private IP to be provided when registering the custom nodes. When setting the default_pod_security_policy_template_id: to restricted Rancher creates RoleBindings and ClusterRoleBindings on the default Namespaces have Network Policies defined Running different applications on the same Kubernetes cluster creates a risk of one compromised application attacking a neighboring application. Network segmentation

(Automated) 1.1.9 Ensure that the Container Network Interface file permissions are set to 644 or more restrictive (Manual) 1.1.10 Ensure that the Container Network Interface file ownership is set to root:root used for users (Manual) 3.2 Logging 3.2.1 Ensure that a minimal audit policy is created (Automated) 3.2.2 Ensure that the audit policy covers key security concerns (Manual) 4.1 Worker Node Configuration host IPC namespace (Automated) 5.2.4 Minimize the admission of containers wishing to share the host network namespace (Automated) 5.2.5 Minimize the admission of containers with allowPrivilegeEscalation (Automated)

Configure Kernel Runtime Parameters Configure etcd user and group Ensure that all Namespaces have Network Policies defined Reference Hardened RKE cluster.yml configuration Reference Hardened RKE Template Namespaces have Network Policies defined Running different applications on the same Kubernetes cluster creates a risk of one compromised application attacking a neighboring application. Network segmentation A network policy is a specification of how selections of pods are allowed to communicate with each other and other network endpoints. Network Policies are namespace scoped. When a network policy is introduced