Private Zone ENI Pod 使用场景 Use cases • Multimedia processing • IoT sensor messages processing • Stream processing at scale • Chat bots • Batch jobs or scheduled tasks • HTTP REST APIs and web

QuickBI DataV Log Service / Dashboard ��� Stream Processing SparkStreaming Function Compute Hadoop HIVE Big Data Analytics Batch Processing Pig PAI MaxCompute EMR Interactive Analytics DLA

Efficient and reliable container release Ø Why not RollingUpdate ? Ø What are the advantages of batch gray release ? • more reliable and better control • More flexible • More efficient StatefulSetPlus Updated Pod OK StatefulSetPlus StatefulSetPlus Batch Gray Release Key Features: Ø Manual/Auto Batch Gray Release Ø Multi-Batch Rollback Ø Multi-Batch InPlaceUpdate Ø Support HPA, CronHPA, VWA (Vertical

Huawei CloudBU Principal Engineer 王雷博 Principal Software Engineer • Huawei(Now) - Cloud Native batch system (Volcano) development • IBM spectrum computing - Cluster resource and workload scheduling reservation p Binpack p Task topology p Zone aware scheduling p … Volcano: A Kubernetes native batch system Gaps for spark Architecture Gaps for spark Architecture 1. Kubectl creates a JobEx object Driver Spark Job2 Executor Executor Executor Executor l How spark on Kubernetes works l Volcano batch system l Use delay pod creation feature to deal with high concurrent job submission l Use queue

2. 命名组是REST路径 /apis/$GROUP_NAME/$VERSION ，并使⽤ apiVersion: $GROUP_NAME/$VERSION （例如 apiVersion: batch/v1 ）。 ⽀持的API组的完整列表可详⻅：Kubernetes API reference 。 使⽤ custom resources 扩展API有两个⽀持的路径： 1. CustomResourceDefinition --runtime-config 来启⽤或禁⽤它们。 -- runtime-config 接受逗号分隔的值。例如：要禁⽤ batch / v1 ，请设置 --runtime-config=batch/v1=false ；想启 ⽤ batch/v2alpha1 ，可设置 --runtime-config=batch/v2alpha1 。 该标志接受逗号分隔的⼀组键值对，键值对描述了 apiserver的运⾏时配置。

进程在正常运行结束后，置为completed状态，且不会再重启，如果进程运行失 败，则根据配置决定是否要重启 # vi xx-job.yml #内容如下 apiVersion: batch/v1 kind: Job metadata: name: myjob #Job名 spec: comple�ons: 3 #一共起1个pod（默认1） ★CronJob控制器 cronjob控制器用于运行周期性pod，定时执行 # vi xx-cronjob.yml #内容如下 apiVersion: batch/v1beta1 kind: CronJob metadata: name: mycronjob #cronJob名 labels: cjlb: xxx #cronjob的标签

����� ��� ����� – �������� ����� ����� ������� ��������� ������� �������� �� ���� Online service Batch jobs Category Online shopping web apps, payment service MR, spark, flink Latency Sensitive Insensitive

DaemonSets, CronJobs? 容器编排的困境 Kubernetes 为我们带来了什么？ 1. 提供了多种多样的基础资源封装：Service，Controllers，ConfigMap，Batch … 2. 自动化的运维机制：HPA，“VPA”，Rollback，Rolling Update 3. 极简的部署方式：YAML + Image Kubernetes 的现状 1. 对于大型应用而言，需要组合使用大量的

strategies and operate our edge devices in a clustered fashion. It really does support distributed processing across devices.” Ben Reif Lead Developer Booz Allen Hamilton 4 www.susergs.com Advancing the strategies and operate our edge devices in a clus- tered fashion. It really does support distrib- uted processing across devices.” Pioneering a New Frontier in Military Tactics As the innovation team at Booz Booz Allen have come to understand, many organiza- tions continue to operate outdated data processing methodologies—struggling to capitalize on the opportunity the edge rep- resents. This has resulted

1min pod is removed from etcd Unhealthy Node Taint/Degrade 1min Node has taints or is degraded Processing Base on the failure reason Unhealth node is healed or removed. Reason classification: Source and Log. 2. Analyze the problem of the node, such as DiskRO, critical Daemonset is not ready. 3. Processing unhealthy node: Heal, Degrade or Isolate. With scoring mechanism and historical operation records