the general trend. How to manage so many clusters ,resources and businesses How to ensure load balancing of cluster nodes 1 2 Improper resource requests 3 Multi-tenant resource preemption How to Business N … How to ensure load balancing of cluster nodes ？ Dynamic-Scheduler Node1 Node2 Kube-scheduler Pod Request Load Level Request Load Level Real Load Level Real Load Level Assigned to Node2 but high load, while some nodes have high resource requests but low load. Dynamic-Scheduler Node1 Node2 Kube-scheduler Pod Request Load Level Request Load Level Real Load Level Real Load Level

automation Cluster management system ● Automation ○ Scheduling ○ Deployment ○ Healing ○ Discovery/load balancing ○ Scaling Scheduling Scheduler Scheduler Scheduling Scheduler Scheduling color=yellow Discovery Select color = yellow color=yellow Discovery yellow.mycluster Select color = yellow Load balancing Controller manager 5 Healing Controller manager 5 Healing Controller manager 5 Healing

Resource Management with vSphere DRS What is DRS? The vSphere Distributed Resource Scheduler (DRS) is a load balancer for VMs deployed on a hypervisor cluster. It has advanced features that can provider actual independent of DRS, but the combination of the two in a cluster is recommended. This will enable load balancing and application of affinity/anti-affinity rules Deploying HA Hosts in an HA cluster are health

Resource Management with vSphere DRS What is DRS? The vSphere Distributed Resource Scheduler (DRS) is a load balancer for VMs deployed on a hypervisor cluster. It has advanced features that can provider actual independent of DRS, but the combination of the two in a cluster is recommended. This will enable load balancing and application of affinity/anti-affinity rules Deploying HA Hosts in an HA cluster are health

health Replicating application instances Using Horizontal Pod Autoscaling Naming and discovering Balancing loads Rolling updates Monitoring resources Accessing and ingesting logs Debugging applications （与3同时），Pod将从该服务的端点列表中删除，并且不再被认为是replication controllers正在运⾏的Pod的⼀部 分。 缓慢关闭的Pod可以继续接收从load balancer转发过来的流量，直到load balancer将其从可⽤列表中移除。 8. 当优雅关闭时间到期时，仍在Pod中运⾏的任何进程都会被SIGKILL杀死。 9. 通过设置优雅关闭时间为0（⽴即删除），Kubelet将在API revision （回滚到之前的Deployment修 订版本）。每次回滚都会更新Deployment的修订版本。 Scale up the Deployment to facilitate more load （扩展Deployment，以便更多的负载） Pause the Deployment （暂停Deployment），从⽽将多个补丁应⽤于其PodTemplateSpec，然后恢复它，开始新

#设置时区 # systemctl restart chronyd # chronyc sources -v ⑦设置网络参数 #首先加载模块 # cat > /etc/modules-load.d/br_ne�ilter.conf < /etc/modules-load.d/k8s-ipvs.conf <load导入；或者使 用内部registry仓库（内部registry镜像仓库里要有以上7个镜像） ★直接使用命令行方式初始化集群 （以下是非HA模式的master初始化，如果要部署高可用集群，则参考第4章）

annotations: service.beta.kubernetes.io/aws-load-balancer-type: "nlb"） 内部及外部Load Balancer 支持（internal LB支持方式 -> annotation: service.beta.kubernetes.io/aws-load-balancer-internal: 0.0.0.0/0） © 2019, Amazon Service load balancer: Network Load Balancer apiVersion: v1 kind: Service metadata: name: nginx namespace: default labels: app: nginx annotations: service.beta.kubernetes.io/aws-load-balancer-type:

Services etcd Pods Endpoints Controller Manager Node Node … Load Balancer Endpoints Controller KubeProxy Pod Node Load Balancer KubeProxy Pod Pod Pod Pod Pod Kubernetes�Service�� Iptables��Service���� 16K LOCs • 200+ �� • ��1.8 Alpha��, Owner @Huawei���1.9�beta • ��ClusterIP�NodePort�External IP�Load Balancer�OnlyLocalNode… • ��iptables�SNAT����� Kubernetes�Service�� Iptables��Service���� ��iptables�������

dhcpOptions/{{ ClusterName }} {{ end }} # --------------------------------------------------------------- # Load Balancer - Masters # # Attach each master ASG to the ELB # ----------------------------------- dhcpOptions/{{ ClusterName }} {{ end }} # --------------------------------------------------------------- # Load Balancer - Masters # # Attach each master ASG to the ELB # -----------------------------------

Flexible and dynamic resource management Dynamic Scheduler is to solve the problem of unbalanced node load in the cluster. Ø Base on current and history node resource usage. Ø Extend Predicate & Priority Ø Reduce container OOM from bursted traffic. Ø Improve cluster resource utilization. Ø If a node load is high, VWA can quickly adjust offline containers‘ cgroup to ensure the SLA of online tasks without