Kubernetes Secrets Agenda • K8s Secrets: Overview • TEE-based K8s Secrets Protection: Solution • Production Experience @ Ant Group • Demo • Summary & Plan K8s Secrets: Overview Background: K8s Secrets Cluster Provider KMS Encryption Provider Background: K8s Secrets • Encryption Keys stored on API Server • Secrets encrypted prior to storage in etcd • Secrets decrypted on API Server prior to use • Encryption Encryption keys stored in a remote KMS • Use envelope encryption scheme • DEK & KEK Motivation: K8s Secrets Protection • Performance & latency • Network • Security • DEK in the clear in memory • Secret

GPU Resource Management On JDOS 梁永清 liangyongqing1@jd.com 提供的服务 1. 用于实验的 GPU 容器 2.基于 Kubeflow 的机器学习训练服务 3.模型管理和模型 Serving 服务 Experiment Training Serving 均基于容器，不对业务方直接提供 GPU 物理机 GPU 实验 JDOS 常规的容器服务

Node Operator: Kubernetes Node Management Made Simple 陈俊(Joe), Ant Financial Agenda • Background and Motivation • Introduction of Operators • Node-Operator • Advanced Topic: • Upgrade Master & Node Components reliably • Canary Rollout • Master & Node Component Versions Management Motivation: Work Order Deployment Worker Order • Upgrade Nodes Versions • Upgrade Node 10.10 Complicated architecture Work order deployment system can not meet the requirements of resource management. Operator Observe Action Analyze • Observe: watch desired resource and actual resource

Turtles all the way down: securely managing Kubernetes secrets with secrets Alexandr Tcherniakhovski, Google Cloud Maya Kaczorowski, Google Cloud Nov 14 2018 Turtles all the way down Turtles @MayaKaczorowski Protecting secrets What’s a secret? Credentials, configurations, API keys, and other small bits of information needed by applications at build or run time Why protect secrets? ● Attractive target in public storage buckets Secret management requirements Identity Require strong identities and least privilege Auditing Verify the use of individual secrets Encryption Always encrypt before

Co-locating helper processes ，促进组合应⽤程序和保留”⼀个应⽤程序的每个容器“模型 Mounting storage systems Distributing secrets Checking application health Replicating application instances Using Horizontal Pod Autoscaling 关于Node的⼀般信息，如内核版本、Kubernetes版本（kubelet和kube-proxy版本）、Docker版本（如果使⽤了Docker 的话）、OS名称。信息由Kubelet从Node收集。 Management（管理） 与 pods 、 services 不同，Node不是由Kubernetes创建的：它是由Google Compute Engine等云提供商在外部创建 的，或存在于物理机或虚 ⽌，除⾮web-0已经Running and Ready。 Pod Management Policies（Pod管理策略） 在Kubernetes 1.7及更⾼版本中，StatefulSet允许您放松其排序保证，同时通过 .spec.podManagementPolicy 字段保留其 唯⼀性和身份保证。 OrderedReady Pod Management（OrderedReady的Pod管理） OrderedReady

Introduction § K8s is a production- grade container orchestration platform § Declarative management of objects using configuration files. § More introductions, go to • K8s official document Endpoints § Ingress § Jobs § Nodes § Namespaces § Pods § Persistent Volumes § Replica Sets § Secrets § Services § Stateful Sets… § K8s通过这些资源模型构建应用程序 § 每一种资源都可以被用户所创建并存储在K8s数 据库中 § 用户通过这些创建这些资源“描绘”应用程序在 Redis Other objects used in OW charts • ConfigMap: like nginx deployment configuration • Secrets: like DB access credentials • Ingress Component Launch Sequence • In Kubernetes, we can use

placement options, for both control plane and worker nodes. 2 levels of scheduling and resource management are active. ​Currently no automatic scheduling integration occurs, that is, Kubernetes is not to solve potential issues with CPU and memory intensive workloads ​Kubernetes default resource management ​How it works ​Extending the functionality of Kubernetes ​Using vSphere DRS with Kubernetes pre-container era Active discussions regarding Kubernetes enhancements going on now in Resource Management Working Group – please join in • See Issue #49964 14 Using a NUMA aware hypervisor to solve

placement options, for both control plane and worker nodes. 2 levels of scheduling and resource management are active. Currently no automatic scheduling integration occurs, that is, Kubernetes is not to solve potential issues with CPU and memory intensive workloads Kubernetes default resource management How it works Extending the functionality of Kubernetes Using vSphere DRS with Kubernetes High pre-container era Active discussions regarding Kubernetes enhancements going on now in Resource Management Working Group – please join in • See Issue #49964 14 Using a NUMA aware hypervisor to solve

public access)  實施角色型存取權控管 (Implement role-based access control)  將 Kubernetes密鑰加密 (Encrypt secrets at rest)  設置 Kubernetes 的許可控制器 (Configure admission controllers)  實施 Kubernetes 網路政策 (Implement Day 1 & Day 2 for k8s clusters Manages access to k8s API for developers IT Operator IaaS Management Internet User Application User Trust Boundary Trust Boundary Trust Boundary Trust Boundary Authentication and Authorization i. Compliance j. File System Permissions k. User Account Management 所有強化在發佈前都經過測試驗證 您不再需要每回合升級都從頭來過 若發現CVE漏洞官方立刻提供修補 •The following servers are not used

its Affiliates. All rights reserved. Amazon Confidential 云端安全工具 Amazon Inspector AWS KMS AWS Secrets Manager AWS WAF AWS IAM Amazon GuardDuty Amazon Macie AWS Security Hub AWS CloudHSM AWS Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential AWS Identity and Access Management (IAM) 身份验证 Kubectl 3) Authorizes AWS identity with RBAC K8s API 1) Passes AWS identity 2)