Sebastian Toader & Zsolt Varga 2021-Feb-26 Apache Kafka with Istio on K8s 2 • Scalability • Resiliency • Security • Observability • Disaster recovery Production grade Apache Kafka on Kubernetes certificate attached automatically by Istio Proxy sidecar container • Client certificate includes the K8s service account of the Kafka client application • SPIFE:///ns//sa/

0 码力 | 14 页 | 875.99 KB | 1 年前

3

glance 185M Number of Active Buyers worldwide 19M Number of Sellers worldwide 1.7B Number of Live Listings $26.6B GMV in Q4 2020 #IstioCon eBay Applications eBay is powered by ● More than 5,000 Application Deployment: Cloud Layout ● Multiple K8s Clusters in an AZ ○ Each K8s cluster ~ 200 - 5,000 nodes ○ Upto 100,000 Pods in a cluster ○ 10,000+ K8s services - including prod, pre-prod, staging worst-case scenario Region R1 AZ 1 AZ 2 AZ n Data Center DC1 K8s Cluster K8s Cluster K8s Cluster K8s Cluster K8s Cluster K8s Cluster Region Rn #IstioCon Application Specs Region

issues. 3. Review the fixes for the issues found in an audit from 2020. 4. Review and improve Istio's fuzzing suite. 5. Perform a SLSA review of Istio. The audit was started with a kickoff meeting, and team time to triage and assess criticality. Results summarised 6 fuzzers written and added to Istio's OSS-Fuzz integration 1 CVE found in Golang 1 vulnerability found that affected Googles managed Istio findings Issue 10 - “H2c handlers are uncapped” - was an interesting finding, in that it affected Googleʼs managed Istio offering, and it led to further investigation that revealed a vulnerability in Golang

Dileo Divya Natesan Andy Olsen Feedback on this project? https://my.nccgroup.com/feedback/67b627f7-a0a2-43b7-ad68-af515a9ed2e0 Executive Summary Synopsis In the summer of 2020, Google enlisted NCC Group no additional cost) worked on the project in tight partnership with Google’s Istio subject matter experts. Scope NCC Group’s evaluation of Istio included: • Istio Architecture: The overall design and the code base shown below: • github.com/istio/istio – 7353c84b560fd469123611476314e4aee553611d • github.com/istio/proxy – c51fe751a17441b5ab3f5487c37e129e44eec823 • github.com/istio/istio.io – 26d

identify problems Iterate • Fix bugs • Repeat Testing starts late in the API development process. That’s not good!! | CONFIDENTIAL Start testing earlier Create and maintain a balanced test pyramid Create services | CONFIDENTIAL 10 Capture API interactions with lua filters Service A Proxy Proxy Service B Service C Proxy Mesh Dynamics Data Store Deploy: kubectl apply -f Capture using Lua filter Service B Service C Proxy req req[A B], trace:r, span:s1 res[A B], trace:r, span:s1 req[B C], trace: r, parent_span: s1 res[B C], trace: r, parent_span: s1 req req[A->B]

Raphael Fraysse @la1nra (Twitter) Github / @lainra Tech Lead, Networking Mercari, Inc. 3 Today’s agenda ● Istio at Mercari ● Stabilizing Istio ● Adopting Istio Istio at Mercari What Is Mercari scale Istio-enabled pods well ○ Use ContainerResource to fix HPA on the application container (From K8S 1.20) ○ Otherwise, add the Sidecar proxy CPU usage into calculation for HPA scale target. ● Exposing ● Then you try to update, and: Error: .LabelSelectorRequirement(nil)}: field is immutable (Since k8s 1.16) 49 Label selector updates for app and version labels Adopting Istio First, headless services

主线程 初始化 日 志 线 程 读 取 配 置 x D S 监 听 网络事件 启 动 工 作 线 程 定时器事件 a d m i n 请 求 X D S 更 新 合 并 s t a t 刷 新 D N S 调度器 工作线程 网络事件 定时器事件 监 听 器 监 听 过 滤 器 释 放 内 存 记 录 s t a t 状 态 更 新 调度器 L 4 网 络 过 滤 Technologies Co., Ltd. All rights reserved. Page 20 生产环境问题分析及解决方法（1） 503 UF问题分析 现象 日志报错503 UF，等待8S后建立连接失败。 日志如下： [2021-02-09T06:29:10.489Z] "GET /v1/xx/xx/xx/xx HTTP/1.1" 503 UF "-" "-" 0 91 288 - "100 接数增加快速恶化 端到端平均时延 降低23%左右 • Envoy: 4线程，4core，默认内存 • fortio –q 0 –c 2~1024连接，http1长 连接模式，每组测试三次，每次30s 测试结果 测试条件 Copyright © Huawei Technologies Co., Ltd. All rights reserved. Page 24 针对Envoy做的一些优化及效果

Vendor logos representation 1. Social Media mentions of presenter and their company a. Keynotes: 2x b. Tech talks, lightning talks, workshops: 1x c. Event supporters: 2x 2. Screensaver / screen between have the logos of all the companies offering swag. 4. Slack / event chat mentions: a. Keynotes: 1x b. Tech talks, lightning talks, workshops: 1x c. Event supporters: 1x 5. One shared mention at slack event activities ● Includes 2 social media mentions, 1 slack mention during the event. ● The sponsor(s) are responsible for the data collection, production and distribution is a responsibility of the sponsoring

• Pilot-Agent只是负责启动S，其他步骤由envoy完成。 • 1. 启动另外一个S进程（Secondary process） • 2. S通知P（Primary process）关闭其管理的端口，由S接管 • 3. S加载配置，开始绑定listen sockets，在这期间使用UDS从P获取合适的listen sockets • 4. S初始化成功，通知P停止监听新的链接并优雅关闭未完成的工作 关闭未完成的工作 • 5. 在P优雅关闭过程中，S会从共享内存中获取stats • 5. 到了时间S通知P自行关闭 • 6. S升级为P • 官方博客：Envoy hot restart什么时候会进行热重启？监控envoy ü获取非正常退出状态 ü抢救机制触发 ü抢救令牌减少一个（总共10个） ü在2(n-1) * 200毫秒后执行（为什么不立即执行） ü失败再次触发抢救机制 ü10 ü10个令牌用完，没有抢救成功，放弃退出优雅关闭envoy ü K8s发送SIGTERM信号让容器优雅关闭 ü Pilot-Agent接收信号通过context关闭子服务，发送SIGKILL关闭envoy ü Envoy不支持优雅关闭，需要通过金丝雀或蓝绿部署方式实现 Envoy优雅关闭实现方式讨论：#3307 #2920Pilot-Discovery——配置中心（PD） uv1版本和v2版本之间的区别

○ Need consistent metrics aggregation ● Traffic management ○ Load balancing for VMs, failover, A/B testing, modern rollouts for VM services ● Security ○ Enforce the same policies in the same way, across onprem register mysql 1.2.3.4 3306 #IstioCon V1.1 Introducing Service Entry Service Entry v.s. Service v.s. Endpoints ● Service Entry ○ An entry that Istio maintains internally ○ Describing the properties External IPs #IstioCon V1.1 ServiceEntry #IstioCon V1.6-1.8 Better VM Workload Abstraction A K8s Service and Pods Two separate object with distinct lifecycles Before Workload Entry, a single Istio