Provider Background: K8s Secrets • Encryption Keys stored on API Server • Secrets encrypted prior to storage in etcd • Secrets decrypted on API Server prior to use • Encryption keys stored in a remote write • LivenessProbe for health check • Configurations • kms-plugin • apiserver • Caching • API server • Set up Encrypted(DEK) => DEK mapping • KMS plugin • Set up SecretKeyName:SecretKeyVersion Annotation: /storage-transform-disable= • Emergency management • High Availability guarantee • KMS • API server & kms-plugin • Cron job backup for KEKs (from KMS) • Static key configuration support in kms-plugin

targetAverageUtilization: 50 • API Object Oriented Programming Core of API “OO” 1.API objects stores in etcd 2.Control loops (Sync Loop) to reconcile API objects Example kubelet SyncLoop proxy proxy 1 Pod created etcd scheduler api-server Example kubelet SyncLoop kubelet SyncLoop proxy proxy 2 Object added etcd scheduler api-server Example kubelet SyncLoop kubelet SyncLoop a node etcd scheduler api-server Example kubelet SyncLoop kubelet SyncLoop proxy proxy 4.1 Detected bind operation 4.2 Start Pod on this machine etcd scheduler api-server Pattern 1: Controller

Highest Level • Container Cluster = “Desired State Management” – Kubernetes Cluster Services (w/API) • Node = Container Host w/agent called “Kubelet” • Application Deployment File = Configuration File be running Worker Node Worker Node Worker Node Kubernetes Master Node (Master & etcd nodes) API K K K App_Y.yaml ContainerImage1 Replicas: 1 ContainerImage2 Replicas: 2 https://youtu.be/PH-2FfFD2PU JSON • Provides core control loops for platform • Watches shared state through apiserver • Makes changes from current to desired • Policy-based workload scheduler • Topology aware • Assists with availability

Operator Pattern 2015.11 2016.12 2017.12 Now K8s 1.1 版本中正式推出 TPR （ThirdPartyResource），首次尝 试解决 K8s API 的扩展性问题， 但存在诸多问题，Alpha 阶段既 夭折 CoreOS 提出 Operator 概念，用 于管理和运行基于应用程序领 域的复杂有状态应用程序。 给出了用 TPR + controller- Pattern 是官方定义的标准扩 展机制，是 K8s Native Application; Operator = CRD + control loop, i.e, Declaretive API + Automation; kubebuilder + controller-runtime + helm Operator Capability Levels Installation 机制获取某种资源的 全量 objects。list 可以简单理解为一个 HTTP GET 请求，watch 为一 个 HTTP/2 长连接 Cache 如何保持与 API Server 一致性 list & watch 机制中，list 获取 API Server 中数据的一份快照，并记 录 ResourceVersion 版本信息，watch 从 ResourceVersion 开始，获取后 续的增量数据。

a standard API allowing a storage provider to write just one plugin that will work for all major container orchestration systems: Kubernetes, Mesos, Docker and Cloud Foundry. Cluster API provider for for vSphere • The Cluster API is a Kubernetes project to bring declarative, Kubernetes-style APIs to cluster creation, configuration, and management. It provides optional, additive functionality on top users looking to try out Kubernetes or develop with it day-to-day. 7 Out of tree components, what changes for me? In-tree vSphere Cloud Provider vSphere CSI Out-of-tree vSphere Cloud Provider

Compute Engine 隨選生成的虛擬機 IaaS and PaaS at Scale Google App Engine #全代管服務 #以容器為基礎 #適合Web應用 #適合Api #全自動擴展+強大的負載平衡 #整合能同步擴展的NoSQL DB Kubernetes ● Kubernetes 是用於自動部署，擴展和管理容器化應用 程序的開源系統 ○ 根據資源需求和其他約束自動放置容器 to use [gcr.io/k8s-cluster-api]: CLI (Installation) Register with Google Cloud Console On-Prem/Public Cloud Provider Any K8s Cluster GCP Connection Proxy K8s API Server Connection Agent End-User Prometheus + Grafana + EFK ● Ingest metrics and logs into Stackdriver without any instrumentation changes ● Aggregate logs from many clusters -- whether GKE or GKE On-Prem Logging and Monitoring Cloud

rowth-for-global-co 企業Kubernetes的關鍵考量因素 1. CNCF符合性 CNCF認證是一項符合性計畫，確保每家廠商的Kubernetes發行版本，都支援所需 的API並提供及時更新。選擇通過CNCF認證的Kubernetes安裝，可協助企業保證 產品的適應性、可預測性及互通性，此外也能避免受到廠商限制，並可提供彈性， 依據功能和需求的演進發展改用替代解決方案。 K3及MicroK8都可在邊緣大幅簡化部署、最佳化及維護Kubernetes的流程。 MicroK8與K3之間的主要差異之一，就是兩者針對Kubernetes API所做的決 定。MicroK8與上游K8 API完全相容，而K3則以較為固定不變的API子集提 供略小一些的二進位檔。 3 2019 2021 2022 2020 1.21.x 1.20.x 1.19.x 1.18.x 1.17 this document may change without notice and Canonical will not be held responsible for any such changes. Canonical Limited, Registered in England and Wales, Company number 110334C Registered Office:

work ● Migrating from ng1 to ng2 (#3152) ● Migrating metrics from Heapster to Kubernetes Metrics API (#2986) ● Apps list page (#2980) Demo Future of Dashboard How do people use Dashboard today kubectl 功能与kubectl保持一致 3. Multi-cluster management 多集群管理 4. Improved security 提高安全性 Top requested changes 1. Third-party plugins or integrations 集成第三方插件 Which third-party plugins or integrations would https://github.com/kubernetes/dashboard/issues /3256#issuecomment-437199403 “Extensibility, modularity, API for libraries to integrate with Dashboard ... it should follow philosophy of K8s, and should be

Endpoints: track backend pod changes <172.17.10.1>:<80> <172.17.10.2>:<80> ... Cluster DNS record: pod1.clusterdomain pod2.clusterdomain ... Service�Endpoints�� Service ���� API Server Services etcd 192.168.60.200:80 -r 172.17.1.2:80 –m # ipvsadm -a -t 192.168.60.200:80 -r 172.17.2.3:80 –m ���� API Server Kube-proxy seesaw Pod Pod Pod Real Server Service IP (Virtual Server) Client netlink

Google Cloud @MayaKaczorowski Protecting secrets What’s a secret? Credentials, configurations, API keys, and other small bits of information needed by applications at build or run time Why protect keys used for encryption of cardholder data, including the following: 3.6.4 Cryptographic key changes for keys that have reached the end of their cryptoperiod (for example, after a defined period of encoded ● A pod can access secrets via the filesystem, as an environment variable, or via Kubernetes API call ● Operations with secrets are audit logged Master kube-apiserver etcd SECRET Kubernetes