Schedule Ceres Job Queue Manager Spark-Operator OfflineJobs Scheduler Kubeflow Hybrid Deploy StatefulSetPlus-Operator Tencent Cloud Mesh MultiCluster-Route-Manager Application & Route Management & Priority Policy. Ø NodeResourceUsage Annotator scores node periodically. Ø Dynamic-Scheduler query node score directly from node to make decisions during predicate and priority stage. Dynamic Scheduler Dockerd self-agent self-agent Patch Node Condition �������������������������� TKEx Web Job Queue Manager Online & Offline Task OfflineTask DynamicQuota Rebalance Worker TKEx-API Message Queue

kubelet can do image GC DevOps Service DevOps Operator DevOps Operator DevOps Service DevOps Manager CronJob k8s API MySQL k8s API MySQL MySQL • Pipeline configuration and history in MySQL • Job Job BuildJob BuildJob BuildJob MySQL MySQL MySQL DevOps Service DevOps Service DevOps Manager Create job Update jobs status to buildjob Submit buildjob List/Watch buildjob Pod Pod Pod Pod Service DevOps Operator Cluster AutoScaler k8s API DevOps Service DevOps Service DevOps Manager Restful API realtime log history log pull metric data ElasticSearch ElasticSearch Prometheus

com/apache-spark-on-k8s/spark l The goal is to bring native support for Spark to use Kubernetes as a cluster manager like YARN, or Mesos. l Spark 2.3 added native support for Kubernetes. l Spark 2.4 added support creation spark-submit \ --master k8s://https://192.168.45.93:5443 \ --deploy-mode cluster \ --name query \ --class com.databricks.spark.sql.perf.BenchmarkQuery \ --conf spark.kubernetes.volcano.enable=true

yellow.mycluster Select color = yellow Load balancing Controller manager 5 Healing Controller manager 5 Healing Controller manager 5 Healing People love automation! I hate Kubernetes! I hate to - Systemctl start kubelet Installation - master - SSH - Install scheduler - Install controller manager - Install API server - Config them correctly - Start them Installation - etcd - SSH - Install etcd -f kube-apiserver.yaml $ kubectl apply -f kube-scheduler.yaml $ kubectl apply -f kube-controller-manager.yaml $ kubectl apply -f kube-proxy.yaml Simplify k8s lifecycle management Manage your cluster with

Node-Exporter KUN-Agent Grafana Blackbox Exporter Prometheus Monitor Manager 微信/邮件 外部探测 Probe Exporter AZ 2 (Local File) Alert Manager gossip Kubernetes 互相监控 AZ 1 Kube-State-Metrics Kubernetes APIServer APIServer cAdvisor Node-Exporter KUN-Agent Alert Manager Prometheus (Local File) Monitor Manager Custom Containers Custom Containers Think in Cloud . 北北京 监控系统⽅方案 • 监控基于 Prometheus 构建，Prometheus 构建，Prometheus 部署于 K8s 集群中，使⽤用 HostPath 存储数据； • Metrics 采集： A. 采集 apiserver、controller-manager、scheduler、etcd、kube-proxy、Kubelet 等组件提供的 metrics B. Kubelet ⾃自带的 cAdvisor 采集容器器 Metrics C. 每个 Node 上以 DaemonSet

etcd ⽤作Kubernetes的后端存储。集群的所有数据都存储在此。请为你Kubernetes集群的etcd数据提供备份计划。 kube-controller-manager kube-controller-manager 运⾏Controller，它们是处理集群中常规任务的后台线程。逻辑上来讲，每个Controller都是⼀ 个单独的进程，但为了降低复杂性，它们都被编译成独⽴的⼆进制⽂件并运⾏在⼀个进程中。 cloud-controller-manager cloud-controller-manager运⾏着与底层云提供商交互的Controller。cloud-controller-manager是在Kubernetes 1.6版中 引⼊的，处于Alpha阶段。 cloud-controller-manager仅运⾏云提供商特定的Controller循环。您必须在kube-controller-manager中禁⽤这些 Controller循环。可在启动kube-controller-manager时将 --cloud-provider 标志设为 external 来禁⽤控制器循环。 cloud-controller-manager允许云供应商代码和Kubernetes内核独⽴发展。在以前的版本中，核⼼的Kubernetes代码依 赖于特定云提供商的功能代码。在未来的版本中，云供应商的特定代码应由云

The Kubernetes Master Node Basic Components Master Node ETCD kube-apiserver kube-controller-manager kube-scheduler • Key/Value Store • Leader based clustering • Can be clustered across Master Nodes The Kubernetes Worker Node Basic Components Master Node ETCD kube-apiserver kube-controller-manager kube-Scheduler Worker Node CRI-containerd Kubeproxy Kubelet • Container Runtime Interface selector to provide a LB and Service DNS ReplicaSets A cluster wide Pod manager providing Pod scaling DaemonSets A Pod manager to ensure a Pod is scheduled across a Cluster Node set StatefulSets Replicated

��! TiDB-Operator ��! TiDB-Operator ��! • tidb-controller-manager! • tidb-scheduler! • tidb-volume-manager! tidb-controller- manager! • �� k8s ���� CRD: TidbCluster, TidbSet! • �� TiDB ����� Controller: • �� k8s scheduler ���� PV ���! • �� PD �������� TiKV ����(����� ����������)! tidb-volume- manager! • �� external-storage �� PV ���! • �� hostPath �� Local PV (StorageClass: pingcap- volume-provisioner)

CipherTrust Transparent Encrypton 的擴展，資料保護可以在每個容器的基礎上應用，兼具保 護容器的內部資料，以及經過容器存取的外部儲存資料， 都統一經由 CipherTrust Manager 集中管理。 優勢 CipherTrust Transparent Encryption for Kubernetes 效益有 : • 合規性 - CipherTrust Transparent 資安解決方案，企業能依據特定用戶、程 序以及容器內的資源組來建立細粒度存取政策。最後， 該解決方案能夠在容器之間建立隔離，所以只有經過授 權的容器才可以存取機敏資料。 CipherTrust Manager CipherTrust Manager 是 CipherTrust Data Security Platform 的核心，包括 CipherTrust 透明加密，可集中管 理平台上所有產品模組的金鑰、安全策略以及日誌管理。 向數 位化轉型時，您可以信賴 Thales 來保護您的有價資料。 關鍵時刻 關鍵技術 Pod 容器集 Pod 容器集 應用程式 使用者 PV Claim CipherTrust Manager 持久儲存 儲存區 PV Controller Storage Class 儲存類別 Kubernetes 節點 CTE for Kubernetes Kubernetes 叢集

attack against K8S with plugin Demo Kubernetes secrets: external secrets Kubernetes Secret manager ... ... ... ... ... Kubernetes secrets: HashiCorp Vault Watch: https://www.youtube.com/watch May be more tightly scoped Additional secret manager logs Depending on secret manager Depending on secret manager In external secret store Kubernetes secrets: summary