Kubernetes 的許可控制器 (Configure admission controllers)  實施 Kubernetes 網路政策 (Implement networking policies)  對容器設置資安規則 (Configure secure context for containers)  分隔敏感的工作負載 (Segregate sensitive workloads) Components) 2. etcd 狀態資料庫 3. 控制平面設置 (Control Plane Configuration) 4. 工作節點 (Worker Node) 5. 政策 (Policies) ©2019 VMware, Inc. 10 Use Cases: Security Architecture Guidance / Replacement for Checklist File System Hardening c. Boot Security d. Process Security e. Minimization of Attack Surface f. Network Security g. Auditing h. Authentication and Authorization i. Compliance j. File System Permissions

Ordinal Index（有序的索引） 对于⼀个有N个副本的StatefulSet，StatefulSet中的每个Pod将被分配⼀个整数序号，范围[0，N），并且唯⼀。 Stable Network ID（稳定的⽹络ID） StatefulSet中的每个Pod，从StatefulSet的名称和Pod的序数派⽣其主机名。 构造的主机名的模式是 $(statefulset name)-$(ordinal) 和删除之前，web-1不会被终⽌。如果web-0失败发⽣在web-2终⽌并完全关闭之后、web-1终⽌之前，web-1将不会终 ⽌，除⾮web-0已经Running and Ready。 Pod Management Policies（Pod管理策略） 在Kubernetes 1.7及更⾼版本中，StatefulSet允许您放松其排序保证，同时通过 .spec.podManagementPolicy 字段保留其 唯⼀性和身份保证。 io/memory-pressure ：Node内存有压⼒。 node.kubernetes.io/disk-pressure ：Node磁盘有压⼒。 node.kubernetes.io/network-unavailable ：Node的⽹络不可⽤。 node.cloudprovider.kubernetes.io/uninitialized ：当kubelet以外部cloud pr

and liveness probe Component Deployment Topology • Use affinity to make deployment topology policies for different component. E.g. controller node and DB node may not be assigned to the same K8s

社区兼容 Kubernetes compatible • Workload: Deployment, StatefulSet, Job, Bare Pod • Pod: Restart Policies, VolumeMounts, Env, InitContainers，Heath check … • Service: LoadBalancer, Headless, Service

want to have a Network object into k8s API • I want a controller to handle add/update/delete of all Network instances • onAdd: create Neutron network • onDelete: delete Neutron network • onUpdate: onUpdate: update Network object status • https://github.com/openstack/stackube/blob/master/pkg/network- controller/network_controller.go Pattern 2: Gode Generator • client-gen: generate typed Kubernetes AP

Amazon Confidential Amazon VPC CNI plugin Elastic network interface Secondary IPs: 10.0.0.1 10.0.0.2 10.0.0.1 10.0.0.2 Elastic network interface 10.0.0.20 10.0.0.22 Secondary IPs: 10.0 / DX Pod Outbound Traffic SNAT EKS worker node Primary elastic network interface Pod Secondary elastic network interface Pod – 100.64. 0.200 © 2019, Amazon Web Services, Inc. or its Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential Service load balancer: Network Load Balancer apiVersion: v1 kind: Service metadata: name: nginx namespace: default labels:

Chaos Mesh 的结构 以 NetworkChaos 为例 ● Controller 向 chaos-daemon 发送请求 ● [Pod network namespace] 设置 ipset 和 iptables ● [Pod network namespace] 设置 qdisc Chaos-daemon loss/delay/dup/c orrupt netem ipset+iptables bandwidth tbf NetworkChaos 实现方法 如何进入目标 Pod 的 Network Namespace ● setns 系统调用 ● nsenter 命令 或在其他进程中 setns ○ 开发、测试更加方便 ○ 使用起来更加简单 ● SideCar 共享 Network Namespace ○ 范围和权限更加可控 Chaos Mesh 使用案例 以 TiDB

compatible with GKE Built for Day 2 Operations PKS simplifies Day 2 operations with built-in network security—powered by NSX, high availability, logging, monitoring, analytics, and automated health K8s-2 n=3 #pks create-cluster K8s-3 n=3 #pks resize K8s-3 n=5 Architecture NSX-T Bosh PKS Admin Network NCP POD 1 POD 4 POD 2 POD 3 POD 5 POD 6 T0 kube-system PODs – Logical Switch Namespace from the Adapter layer • NSX API Client: Implements a standardized interface to the NSX API Network Container Plugin (NCP) NSX Manager Kubernetes Master etcd API-Server Scheduler NSX Container

--kubernetes- version=v1.19.4 \ --apiserver-adver�se- address=10.99.1.51 \ # api server地址 --pod-network-cidr=10.244.0.0/16 \ # pod容器网段 --service-cidr=10.7.0.0/16 \ # service网段，即cluster ip网段 --ignor --kubernetes- version=v1.28.2 \ --apiserver-adver�se- address=10.99.1.51 \ # api server地址 --pod-network-cidr=10.244.0.0/16 \ # pod容器网段 --service-cidr=10.7.0.0/16 \ # service网段，即cluster ip网段 --ignor h�ps://limaofu.github.io/scripts/kube-flannel-v0.13.0.yml # vi kube-flannel.yml #将里面的net-conf.json下面的Network网段改为规划的pod网段 #默认使用的docker镜像是quay.io/coreos/的仓库，可改为自己集群的docker仓库 保存，退出 # kubectl apply -f kube-flannel

•NetworkScope Provision •OS •Flavor •ComputeNode Configuration •Kernel params •Environment config •Network Kubernetes •Core components •Addon •Taint Operations Our thinking of datacenter modeling by K8sAddons, K8sDeployment KafkaCluster, HadoopCluster, MongoDB, ESCluster …… Fleet (Compute, Network, Storage) Configuration Management Infrastructure Service Application Service Recap We are