GPU Resource Management On JDOS 梁永清 liangyongqing1@jd.com 提供的服务 1. 用于实验的 GPU 容器 2.基于 Kubeflow 的机器学习训练服务 3.模型管理和模型 Serving 服务 Experiment Training Serving 均基于容器，不对业务方直接提供 GPU 物理机 GPU 实验 JDOS 常规的容器服务

Node Operator: Kubernetes Node Management Made Simple 陈俊(Joe), Ant Financial Agenda • Background and Motivation • Introduction of Operators • Node-Operator • Advanced Topic: • Upgrade Master & Node Components reliably • Canary Rollout • Master & Node Component Versions Management Motivation: Work Order Deployment Worker Order • Upgrade Nodes Versions • Upgrade Node 10.10 Complicated architecture Work order deployment system can not meet the requirements of resource management. Operator Observe Action Analyze • Observe: watch desired resource and actual resource

Very manual, no fault tolerance, hard to scale, etc • Scheduling, provisioning, and resource management of multiple containers – Docker, Mesos à Kubernetes Support – AWS, Azure, Google à Kubernetes ContainerImage2 Replicas: 2 Kubernetes 101 at the Highest Level • Container Cluster = “Desired State Management” – Kubernetes Cluster Services (w/API) • Node = Container Host w/agent called “Kubelet” • Application Kubernetes Clusters Desired state of Application The difference between PKS and Kubernetes Open Source Project – Google/Pivotal/VMware 21 Container scheduling, scale, resiliency, and Day 2 Desired state of

Application Catalog | Monitoring | Logging Management Plane Infrastructure Services - Policy Management - Cluster Operations - User Management - Lifecycle Management Infrastructure Services (Networking and install the latest version of apiserver-builder • Create project path in your GOPATH • Go into your project path and init your project ‘your-domain’ would be like your private tenant name. • Then

Chair of Kubernetes VMware SIG. GitHub: @cantbewong Software Engineer VMware First open source project was to enable GPU on Kubernetes with vSphere. Also actively contributing to kubelet, device manager placement options, for both control plane and worker nodes. 2 levels of scheduling and resource management are active. ​Currently no automatic scheduling integration occurs, that is, Kubernetes is not to solve potential issues with CPU and memory intensive workloads ​Kubernetes default resource management ​How it works ​Extending the functionality of Kubernetes ​Using vSphere DRS with Kubernetes

Version Control sync / watch clean history jobs Basic Concepts（partial） Repository Managed Project Pipeline / Stage / Task Dockerfile / Scripts Common Configuration ConfigMap/Secret Data Volume credential using secret - resources Memory / CPU / GPU Data cache CI/CD Examples - Artifact Management user scripts using ConfigMap Job - pod template • upload files to storage service once user • Future plan Our Future Plan • More task templates to be added, integrate more CI/CD and project management tools • Optimize UI generation methodology • Improve development experience, such as CLI

��� ����� �� – Jian He – Staff Engineer @Alibaba cluster management team – Staff Engineer @Hortonworks – Hadoop Committer & Project Management Committee member – Bushuang Gao – Senior Engineer @Alibaba NODE Online service Console Offline jobs L&W L&W GRPC RPC: VTRON RPC: VTRON RPC Resource management VTRON: Virtual Total Resources Of Node cgroup �������� ������� Kubernetes YARN Online service

coupling the kube-controller-manager to cloud- provider specific code. In order to free the Kubernetes project of this dependency, the cloud-controller-manager was introduced. CSI provider for vSphere • Container provider for vSphere • The Cluster API is a Kubernetes project to bring declarative, Kubernetes-style APIs to cluster creation, configuration, and management. It provides optional, additive functionality on version 1.13) and will graduate to Stable/GA in a couple of releases. Status within the Kubernetes project 9 Moving out of tree: the CSI Provider Why it exists Handles C/R/U/D of storage volumes Coordinate

• Internal systems or commercial software Kubernetes • The container orchestration and management project created by Google • Successor of Google Borg/Omega system • One of the most popular open • kubelet -> gRPC -> dockershim -> dockerd • go2idl: gogoprotobuf based protobuf gen CRI Management kubelet Workloads Orchestration kubelet SyncLoop Scheduling api-server Etcd bind pod, node

microservices-centric strategy, in solving the most complex of infrastruc- ture management challenges. When it came to Kubernetes management, the team trialed a number of options. “K3s has been a foundational heterogeneous solution. It was devel- oped in recognition of the diverse range of hardware in the field—a project might run in AWS, Azure or GCP (or a mixture), and so the SmartEdge infrastructure had to support