Practice — 王磊磊 @TenxCloud Agenda • Our DevOps Expectations • Kubernetes Capabilities/Advantages to Build DevOps Solution • Architecture and Features • CRD and operator design • Pipeline / Stage/ Task availability • Extensibility / Integration • CI/CD examples • Future plan Our DevOps Expectations • Build a platform and easy to integrate with other DevOps/third-party tools • Easy to be customized as Kubernetes Capabilities/Advantages to Build DevOps Solution Pod Job CronJob • k8s itself is NOT a PaaS or DevOps platform，but … • k8s resources that can be used to build DevOps solution Volumes ConfigMap

Pool 统一的模型层 平台统一“能力池” 模块化的交付系统 - GitOps “应用”配置 Git (as source of truth) 持续集成 ● Build ● Run Unit Tests ● Build Docker Image ● Push Docker Image Image Registry Operational Configs (YAML) Revision 的应用模型 • 围绕 GitOps 的持续交付 = “以应用为中心”的 K8s KubeVela Git (as source of truth) 持续集成 ● Build ● Run Unit Tests ● Build Docker Image ● Push Docker Image Image Registry AutoScaling Controller Rollout com/180074935/channel/detail?cid=138178 KubeVela demo - KubeVela Cli 整体能力介绍 - Getting started/Application/Traits/System/Capability - 应用创建 - by OAM: https://github.com/zzxwill/try-cloudnative/tree/master/cloudnativeto-presentation-

Scalable Kubernetes Applications • Scalable Infrastructure for Applications Application Operating System Physical Infrastructure Platform Containers as Enabler Fast Boot Environments Rapidly Portable Needed Application Operating System Physical Infrastructure Containers and VMs - A Practical Comparison Containers Containers virtualize the operating system limiting the the number of application applications on the same OS Allows you to run multiple OS on the same hardware Application Operating System Physical Infrastructure Containers VMware Hypervisor VMs Docker Containers User Cases 9

container orchestration and management project created by Google • Successor of Google Borg/Omega system • One of the most popular open source projects in this world • Written by, and heavily depends runtime=remote --container-runtime-endpoint=/var/run/xxx.sock -- feature-gates=AllAlpha=true"' /etc/systemd/system/kubelet.service.d/10- kubeadm.conf • kubeadm init • kubeadm join --token $token ${master_ip:port} containers • re-use images • well-designed architecture for your container workloads • “How can I build distributed micro-services with container?” Programming Pattern • Sidecar apiVersion: v1 kind:

File System Hardening c. Boot Security d. Process Security e. Minimization of Attack Surface f. Network Security g. Auditing h. Authentication and Authorization i. Compliance j. File System Permissions Harbor對於容器映像的安全防護措施 Development Team RBAC UAA AUTH REPL Image Pull K8s Cluster deployed by PKS Build Image Push Image Scan Image For CVEs Sign Trusted Image kubectl run Image Registry Image

application Shared state in Redis and SQL database Scalable userfacing API in GoLang Queuing system Application technologies Best of breed-technologies Right tool for the right job Easy prototyping Deployment, CI and CD Local development Gitlab runner Dockerhub Pod Pod Pod -Build -Test -Push to dockerhub -Deploy Application infrastructure The services around Kubernetes Take

Credentials, configurations, API keys, and other small bits of information needed by applications at build or run time Why protect secrets? ● Attractive target ○ Controls access or use of sensitive resources Separate where secrets are used vs managed Encryption at different layers (or turtles) disks file system etcd Recommendation: Use two-layers of encryption, e.g., full-disk & application-layer … then

github.com/GoogleContainerTools/jib Build containers faster with Jib A container image builder for Java applications Our Team Cloud Tools for Java Appu Goundan @coollog @loosebazooka Qingyang ndencies to target/dependencies/ Some more searching github.com/GoogleContainerTools/jib ... <build> com.spotify dockerfile-maven itory> ${project.version} build> ... github.com/GoogleContainerTools/jib What did we do? 1. Write first Dockerfile 2. Reduce

Rancher Labs, Inc. API Aggregation • What API aggregation provides • Extended with additional APIs • Build your own API server • Requirements of aggregation layer • Running Kubernetes 1.7 Cluster • Enable apiserver flags © 2017 Rancher Labs, Inc. Setup an Extension API Server • Use apiserver-builder to build your own API server • https://github.com/Kubernetes-incubator/apiserver-builder • Download and install Then initialize your own resource group, version and kind. • Your API server could be build and run now • Build as an image and run in a cluster © 2017 Rancher Labs, Inc. API Server Aggregation Architecture

配置版本，发布，回滚，可以更加方便 微服务的开发阶段 统一采用gRPC协议和protobuf编解码 CI check 阶段 • 主要做 pb 的 format、lint、breaking 检查。 CI build 阶段 • 会基于 pb 的注释自动产生文档，并推送至内部的微服务管理系统接口平台中 • 会生成 Go/PHP/Node/Java 桩代码和错误码，推送到指定的仓库 开发阶段 • go get https://ego.gocn.vip/ micro/chapter1/build.ht ml 微服务的部署阶段 注入信息 版本信息 发布版本 • 执行./bin/hello --version • 查看线上使用框架版本 https://ego.gocn.vip/ micro/chapter1/build.ht ml 微服务的部署阶段 注入信息 版本信息 发布版本 • 配置 管理版本信息 管理拓扑关系 管理成本 资料 框架：https://github.com/gotomicro/ego 编译：https://ego.gocn.vip/micro/chapter1/build.html 链路：https://ego.gocn.vip/micro/chapter2/trace.html 限流：https://ego.gocn.vip/frame/client/sentinel