cgroupdriver=systemd" ] } # mkdir -p /etc/systemd/system/docker.service.d # docker info ★docker会修改防火墙规则，导致pod网络不通 # vi /usr/lib/systemd/system/docker.service #在[Service]下的ExecStart=/usr/bin/dockerd #安装k8s二进制组件 （<=1.23版本） # systemctl enable kubelet # systemctl start kubelet ③k8s集群初始化 # kubeadm version #先查看k8s版本 # GitVersion:"v1.19.4" # kubeadm config images list #查看k8s其他组件的docker镜像名，默认用 7个镜像） ★直接使用命令行方式初始化集群 （以下是非HA模式的master初始化，如果要部署高可用集群，则参考第4章） kubeadm init --kubernetes- version=v1.19.4 \ --apiserver-adver�se- address=10.99.1.51 \ # api server地址 --pod-network-cidr=10.244

Architecture and Features • CRD and operator design • Pipeline / Stage/ Task / Task Template / Version Control • Logging, monitoring, autoscaling, high availability • Extensibility / Integration • CI/CD • Architecture and Features • CRD and operator design • Pipeline/Stage/Task/Task Template/Version Control/UI generation/Volume... • Logging, monitoring, autoscaling, high availability • Extensibility/Integration history in MySQL • Logging in central logging service - ElasticSearch • Metric data in monitoring system - prometheus • Alertmanager to invoke various alert and related actions docker registry Kubernetes

Xiang Li xiang.li@coreos.com | Head of distributed system Self driving infrastructure Topics ● Cluster management systems ● Today’s problems with operating cluster management systems ● A self-driving components ○ dynamic dependencies ○ fast deployment iteration ● Solution: automation Cluster management system ● Automation ○ Scheduling ○ Deployment ○ Healing ○ Discovery/load balancing ○ Scaling Scheduling cluster Need an initial control plane to bootstrap a self-hosted cluster Bootkube: ● Acts as a temporary control plane long enough to be replaced by a self-hosted control plane. ● Run only on very

Label 允许⽤户随⼼所欲地组织他们的资源。Annotation 允许⽤户使⽤⾃定义信息来装饰资源以⽅便他们的⼯作流程， 并为管理⼯具提供检查点状态的简单⽅法。 此外， Kubernetes control plane 所⽤的API 与开发⼈员和⽤户可⽤的API相同。⽤户可以使⽤ their own API 编写⾃⼰ 的控制器，例如 scheduler ，这些API可由通⽤ command-line Dashboard 执⾏： kubectl proxy 02-安装单机版Kubernetes 8 访问： http://localhost:8001/api/v1/namespaces/kube-system/services/https:kubernetes-dashboard:/proxy/#!/overview? namespace=default 参考： https://kubernetes install jinja2 --upgrade 如果执⾏ pip2 install jinja2 --upgrade 出现类似如下的提示： You are using pip version 9.0.1, however version 18.0 is available. You should consider upgrading via the 'pip install --upgrade pip'

also supports an underlying tier of high availability and automated placement options, for both control plane and worker nodes. 2 levels of scheduling and resource management are active. ​Currently g. Oracle, MongoDB), present a workload which will attempt to detect and consume as much of the system’s memory as possible. Where does this lead? Node 0 32GB Node 1 21GB 2 CPU Nodes – NUMA host performance effects. (e.g interleaving get predictable albeit reduced performance) • A cgroup aware version (e.g. Java jre v10) can be deployed • This is often not available – many were developed in a

also supports an underlying tier of high availability and automated placement options, for both control plane and worker nodes. 2 levels of scheduling and resource management are active. Currently no g. Oracle, MongoDB), present a workload which will attempt to detect and consume as much of the system’s memory as possible. Where does this lead? Node 0 32GB Node 1 21GB 2 CPU Nodes – NUMA host performance effects. (e.g interleaving get predictable albeit reduced performance) • A cgroup aware version (e.g. Java jre v10) can be deployed • This is often not available – many were developed in a pre-container

©2019 VMware, Inc. 7  關閉公開存取 (Disable public access)  實施角色型存取權控管 (Implement role-based access control)  將 Kubernetes密鑰加密 (Encrypt secrets at rest)  設置 Kubernetes 的許可控制器 (Configure admission controllers) container images)  開啟稽核日誌 (Enable audit logging)  跟上最新的 Kubernetes版本 (Keep your Kubernetes version up to date) Kubernetes Security Best Practices Kubernetes安全性的最佳實務指導 資料來源: https://blog.sqreen org/benchmark/kubernetes/ 控制措施 如何查核 如何查核 參考資訊 預設配置 原因理由 如何查核 1. 控制平面元件 (Control Plane Components) 2. etcd 狀態資料庫 3. 控制平面設置 (Control Plane Configuration) 4. 工作節點 (Worker Node) 5. 政策 (Policies) ©2019

Tencent’s business by using kubernetes native approach. • Adapt to various internal systems like Route System, CMDB, CI, Security Platform, etc. • Declarative application lifecycle management. • Support big RollingUpdate ? Ø What are the advantages of batch gray release ? • more reliable and better control • More flexible • More efficient StatefulSetPlus StatefulSetPlus Service (Kube-proxy, CLB, etc (Vertical Workload Autoscaler) Ø Keep share memory during Pod upgrade Ø Scaled Up with LGV (Last Good Version) Ø Per Pod Per PV Ø Per Workload Per PV Ø Pod Auto Migrate when Node Abnormal Ø Gray Release

configure, and manage clusters in GKE and GKE On-Prem ● Cluster environments are consistent (k8s version, OS image, plug-ins, components configuration) Orchestrate and manage on-prem containers just installation ● Private container registry support ● Latest 3 versions of k8s ● High-availability control plane ● Auto-repair Installation and Configuration $ gke-on-prem create cluster --dry-run Welcome want to install your cluster? [1] vSphere v6.5 Please enter your numeric choice [1]: 1 What version of GKE On-Prem do you want to install? [1] 1.10.3 (Uses k8s v1.10.3) [2] 1.9.2-rc2 (Uses K8s 1.9

Scalable Kubernetes Applications • Scalable Infrastructure for Applications Application Operating System Physical Infrastructure Platform Containers as Enabler Fast Boot Environments Rapidly Portable Needed Application Operating System Physical Infrastructure Containers and VMs - A Practical Comparison Containers Containers virtualize the operating system limiting the the number of application applications on the same OS Allows you to run multiple OS on the same hardware Application Operating System Physical Infrastructure Containers VMware Hypervisor VMs Docker Containers User Cases 9