GPU Resource Management On JDOS 梁永清 liangyongqing1@jd.com 提供的服务 1. 用于实验的 GPU 容器 2.基于 Kubeflow 的机器学习训练服务 3.模型管理和模型 Serving 服务 Experiment Training Serving 均基于容器，不对业务方直接提供 GPU 物理机 GPU 实验 JDOS 常规的容器服务

Node Operator: Kubernetes Node Management Made Simple 陈俊(Joe), Ant Financial Agenda • Background and Motivation • Introduction of Operators • Node-Operator • Advanced Topic: • Upgrade Master & Node Components reliably • Canary Rollout • Master & Node Component Versions Management Motivation: Work Order Deployment Worker Order • Upgrade Nodes Versions • Upgrade Node 10.10 Complicated architecture Work order deployment system can not meet the requirements of resource management. Operator Observe Action Analyze • Observe: watch desired resource and actual resource

Very manual, no fault tolerance, hard to scale, etc • Scheduling, provisioning, and resource management of multiple containers – Docker, Mesos à Kubernetes Support – AWS, Azure, Google à Kubernetes ContainerImage2 Replicas: 2 Kubernetes 101 at the Highest Level • Container Cluster = “Desired State Management” – Kubernetes Cluster Services (w/API) • Node = Container Host w/agent called “Kubelet” • Application Switch Namespace ‘foo’ PODs – Logical Switch Namespace ‘demo’ PODs – Logical Switch Cluster Management Nodes – Logical Switch Master ‘VM’ etcd NCP API Srv Worker ‘VM’ Pod 1 Pod 2 Worker ‘VM’

America December 13, 2018 2 Open Source Community Relations Engineer VMware Active in Kubernetes storage community since 2015. Chair of Kubernetes VMware SIG. GitHub: @cantbewong Application Platform placement options, for both control plane and worker nodes. 2 levels of scheduling and resource management are active. Currently no automatic scheduling integration occurs, that is, Kubernetes is not to solve potential issues with CPU and memory intensive workloads Kubernetes default resource management How it works Extending the functionality of Kubernetes Using vSphere DRS with Kubernetes High

Unit November 12, 2018 2 Open Source Community Relations Engineer VMware Active in Kubernetes storage community since 2015. Chair of Kubernetes VMware SIG. GitHub: @cantbewong Software Engineer VMware placement options, for both control plane and worker nodes. 2 levels of scheduling and resource management are active. ​Currently no automatic scheduling integration occurs, that is, Kubernetes is not to solve potential issues with CPU and memory intensive workloads ​Kubernetes default resource management ​How it works ​Extending the functionality of Kubernetes ​Using vSphere DRS with Kubernetes

Kubernetes满⾜了在⽣产中运⾏的应⽤程序的⼀些常⻅需求，例如： Co-locating helper processes ，促进组合应⽤程序和保留”⼀个应⽤程序的每个容器“模型 Mounting storage systems Distributing secrets Checking application health Replicating application instances Using 关于Node的⼀般信息，如内核版本、Kubernetes版本（kubelet和kube-proxy版本）、Docker版本（如果使⽤了Docker 的话）、OS名称。信息由Kubelet从Node收集。 Management（管理） 与 pods 、 services 不同，Node不是由Kubernetes创建的：它是由Google Compute Engine等云提供商在外部创建 的，或存在于物理机或虚 piserver⼀个 --runtime-config 选项来禁⽤StatefulSet。 给定Pod的存储必须由 PersistentVolume Provisioner 根据请求的 storage class 提供，或由管理员预先设置。 删除/缩容StatefulSet将不会删除与StatefulSet关联的Volume。这样做是为了确保数据安全性，这通常⽐⾃动清除 StatefulSet所有相关资源更有价值。

Monitoring | Logging Management Plane Infrastructure Services - Policy Management - Cluster Operations - User Management - Lifecycle Management Infrastructure Services (Networking, Storage, DNS, Load Balancer Server Service Instance-1 Instance-2 Instance-3 Expose service & registry API ETCD/ Other Storage © 2017 Rancher Labs, Inc . Thanks! Yuxing 2017-8-5

Background: K8s Secrets • Encryption Keys stored on API Server • Secrets encrypted prior to storage in etcd • Secrets decrypted on API Server prior to use • Encryption keys stored in a remote KMS complicated! ü User access management => raw and extensive! ü Secrets management => crucial! • Financial-grade security [1] KubeCon China 2018: Node Operator: Kubernetes Node Management Made Simple - Joe Chen ) • Gray release • extensions-webhook: /mutating-secret • Annotation: /storage-transform-disable= • Emergency management • High Availability guarantee • KMS • API server & kms-plugin • Cron job

information - volumes Storage Service rbd / nfs / glisters pvc pvc pvc registry credential using secret - resources Memory / CPU / GPU Data cache CI/CD Examples - Artifact Management user scripts using ConfigMap Job - pod template • upload files to storage service once user build completes - volumes Storage APIs user build task • build the application package init task • prepare code repository repository sidecar build task lifecycle - preStop - volumes storage config using secret Query artifact data DevOps Operator Manage the Job CI/CD Examples - Human/Manual Task system email config

Github ○ Accessible by users who shouldn’t have access, e.g., CEO ○ Stored in public storage buckets Secret management requirements Identity Require strong identities and least privilege Auditing security against penetration. Similarly, poor key management may easily compromise strong algorithms.” NIST SP 800-57, Recommendation for Key Management Keys get old Key rotation ● Key rotation is meant stored cardholder data against disclosure and misuse. 3.6 Fully document and implement all key-management processes and procedures for cryptographic keys used for encryption of cardholder data, including