name: traefik-dashboard-route spec: entryPoints: - web routes: - match: Host(`traefik-dashboard.xxx.com`) kind: Rule services: - name: traefik name: kubernetes-dashboard-tls spec: entryPoints: - websecure routes: - match: Host(`k8s-dashboard.xxx.com`) kind: Rule services: - name: kubernetes-dashboard ceph-dashboard-tls namespace: rook-ceph spec: entryPoints: - websecure routes: - match: Host(`ceph-dashboard.xxx.com `) kind: Rule services: - name: rook-ceph-mgr-dashboard

离不同环境业务，通过特定标识来识别业务线。 k8s-service k8s-dns注册服务名，通过配置文件关键字关联业务线应用名称，保持应用和k8s之间的关联。 k8s-app-name 容器host应用名称，deployment 名，通过配置文件关键字关联业务线应用名称，保持应用和k8s之间的关联。 规范 范例 应用名称 ai-dc-server ai-dc-web ai-dc-api Flannel负责在容器集群中的多个节点之间提供第3层IPv4网 络。 工作模式： 1.vxlan 通过封装协议解包收发包mtu1450，vxlan可以在分 布多个网段的主机间构建2层虚拟网络 。 2.host-gw 通过宿主机路由同步收发包，必需工作在二层。 1.系统启动，flanneld下发docker子网配置，docker启动获 取子网配置生成docker0 生成gateway网卡; 2.no tcp延迟:calico-bgphosthost-gwcalico-ipip>flannel_host-gw>calico-bgp>host 带宽：host>calico-bgp>flannel_host-gw>flannel-vxlan>calico-ipip host:指物理机直连网络 calic

此时，会看到 inventory/mycluster/host.ini ⽂件内容类似如下： [k8s-cluster:children] 03-使⽤Kubespray部署⽣产可⽤的Kubernetes集群（1.11.2） 12 kube-master kube-node [all] node1 ansible_host=172.20.0.88 ip=172.20 0.88 node2 ansible_host=172.20.0.89 ip=172.20.0.89 node3 ansible_host=172.20.0.90 ip=172.20.0.90 node4 ansible_host=172.20.0.91 ip=172.20.0.91 node5 ansible_host=172.20.0.92 ip=172.20.0 io/google_containers/liveness livenessProbe: httpGet: # when "host" is not defined, "PodIP" will be used # host: my-host # when "scheme" is not defined, "HTTP" scheme will be used

"ClickHouseInstallation" metadata: name: "demo-01" spec: configuration: zookeeper: nodes: - host: zookeeper-0.zookeepers.zoo1ns clusters: - name: "demo-01" layout: shardsCount: Можно собрать желаемую конфигурацию ClickHouse host>::host> host>0.0.0.0host> 1

App 3 Bins/Libs Container Engine Docker Host What is Kubernetes? 13 OS App 1 Bins/Libs App 2 Bins/Libs App 3 Bins/Libs Container Engine Docker Host Kubernetes Slave OS App 1 Bins/Libs App 3 Bins/Libs Container Engine Docker Host Kubernetes Slave OS App 1 Bins/Libs App 2 Bins/Libs App 3 Bins/Libs Container Engine Docker Host Kubernetes Slave Kubernetes Master P1R3 P2R2 Container Cluster = “Desired State Management” – Kubernetes Cluster Services (w/API) • Node = Container Host w/agent called “Kubelet” • Application Deployment File = Configuration File of desired state • Container

components ● Core components deployed as native API objects Self-hosted k8s Architecture Why Self-host Kubernetes? ● Operational expertise around app management in k8s extends to k8s itself ○ E.g. scaling Kubernetes directly translate to improvements in managing Kubernetes Simplify Node Bootstrap On-host requirements become: ● Kubelet ● Container Runtime (docker, rkt, …) Any Distro Node Bootstrap ● Install container runtime ○ $pkgmanager install [docker|rkt] ● Write kubeconfig ○ scp kubeconfig user@host:/etc/kubernetes/kubeconfig ● Start kubelet ○ Systemctl start kubelet $ kubectl apply -f kube-apiserver

system’s memory as possible. Where does this lead? Node 0 32GB Node 1 21GB 2 CPU Nodes – NUMA host When Linux initially allocates a threads, it is assigned a preferred node, by default memory allocations B (Worker) VM K8S Prod (Master) (Worker) VM K8S Prod (Worker) (Worker) (VM Anti-Affinity) Host-VM Rules 24 Extending Kubernetes with vSphere HA What is HA? A least 2 hypervisor hosts are required an HA cluster are health monitored and in the event of a failure, the virtual machines on a failed host are restarted on alternate hosts. When running on hardware that supports health reporting, Pro-active

system’s memory as possible. Where does this lead? Node 0 32GB Node 1 21GB 2 CPU Nodes – NUMA host When Linux initially allocates a threads, it is assigned a preferred node, by default memory allocations B (Worker) VM K8S Prod (Master) (Worker) VM K8S Prod (Worker) (Worker) (VM Anti-Affinity) Host-VM Rules 25 Extending Kubernetes with vSphere HA What is HA? A least 2 hypervisor hosts are required an HA cluster are health monitored and in the event of a failure, the virtual machines on a failed host are restarted on alternate hosts. When running on hardware that supports health reporting, Pro-active

TCP_RR(r/s) TCP_CRR(r/s) Overlay方案性能 host vxlan ipip gateway 23540 8368 22127 7675 21007 7231 0 5000 10000 15000 20000 25000 TCP_RR(64) TCP_CRR(64) Underlay方案性能 Host Bridge NAT IPIP+Gateway混合Overlay方案 y方案 •短链接IPIP包量比Vxlan多14.1% •Gateway比Vxlan多40.5% •方案被Flannel社区合并 Underlay方案 • Bridge方式仅比Host差6%，一般 overlay比Host差20~40% • SRIOV方式比Bridge CPU下降38.3%， 包量+6% 性能 Docker、Docket、Gaiastack P2P Agent下载镜像对比