#IstioCon Using ECC Workload Certificates (pilot-agent environmental variables) Jacob Delgado / Aspen Mesh #IstioCon ECC workload certificates ● In various environments, the need for x509 certificates cryptography (using ECDSA P-256) to use this feature ● Only ECDSA P-256 is supported #IstioCon pilot-agent environmental variables Disclaimer: Environmental variables and their use are considered experimental the ECC_SIGNATURE_ALGORITHM environmental variable on sidecar ejection to ECDSA for use by pilot-agent ○ For gateways this environmental variable also must be set on installation/upgrade #IstioCon

across globe peering with the Internet closer to the customer ○ PoPs are mini AZs Region R1 AZ 1 AZ 2 AZ n Data Center DC1 Region Rn #IstioCon Application Deployment: Cloud Layout ● Multiple K8s Capability to run all applications from a single region or AZ in a worst-case scenario Region R1 AZ 1 AZ 2 AZ n Data Center DC1 K8s Cluster K8s Cluster K8s Cluster K8s Cluster K8s Cluster K8s Network Load-balancers, etc. ■ Full isolation by confining service failures to AZ boundary AZ 1 AZ 2 AZ n Data Center DC1 K8s Cluster K8s Cluster K8s Cluster K8s Cluster K8s Cluster K8s

Envoy原理介绍及线上问题踩坑 介绍人：张伟 Copyright © Huawei Technologies Co., Ltd. All rights reserved. Page 2 个人介绍 张伟 华为云容器网格数据面技术专家 拥有10年以上中间件及高性能系统开发经验， 作为架构师及核心开发人员发布过传输网管系 统、Tuxedo交易中间件、ts-server多媒体转码服 设计及开发工作。 Copyright © Huawei Technologies Co., Ltd. All rights reserved. Page 3 目录 1. Envoy启动及配置文件 2. Envoy流量拦截原理、常用部署方式 3. Envoy可扩展过滤器架构、可观测性 4. Envoy线程模型 5. 生产环境问题分析及解决方法 6. 针对Envoy做的一些优化及效果 7. 应用来说完全透明。支持主要常用网路协议 Http1/Http2/Tls/gRPC/Tcp等。 Copyright © Huawei Technologies Co., Ltd. All rights reserved. Page 6 Envoy原理及总体架构-启动 istiod Pilot-agent Pilot-agent apiServer iptables iptables

(CC BY 4.0) Istio Security Audit, 2023 Table of contents Table of contents 1 Executive summary 2 Notable findings 3 Project summary 4 Audit scope 6 Overall assessment 7 Fuzzing 9 Threat model Formalise a threat model of Istio to guide the security audit as well as future security audits. 2. Carry out a manual code audit for security issues. 3. Review the fixes for the issues found in an audit errors ● 1 case of using a deprecated library ● 1 race condition 2 Istio Security Audit, 2023 Notable findings Issue 10 - “H2c handlers are uncapped” - was an interesting finding, in that it affected

APP logfile Kubernetes console APP logfile APP logfile APP logfile Kubernetes console DC2 DMZ Intranet APP logfile APP logfile APP logfile Kubernetes console APP logfile APP logfile logfile Kubernetes console search &analysis Prometheus TSDB基于请求和日志的关联性改进架构 A Agent B Agent C Agent Request(Transaction ID) A(application) Trasanctionid（CA SDK support） TOM (who) Create transaction ID Request(Transaction ID)Java探针的基本原理 A.class 1 2 3 4 5 8 9 Request Response JVM 6 10 7 Class Loader Engine Agent A’.class JavaAgent 监控数据暂 存区 运行时数据区如何基于Istio的现有组件去实现 Kubernetes

Natesan Andy Olsen Feedback on this project? https://my.nccgroup.com/feedback/67b627f7-a0a2-43b7-ad68-af515a9ed2e0 Executive Summary Synopsis In the summer of 2020, Google enlisted NCC Group to perform configuration options that Istio believes match security best practices. See Appendix B on page 40. 2 | Google Istio Security Assessment Google / NCC Group Confidential Dashboard Target Metadata Engagement issues 2 Total issues 18 Category Breakdown Access Controls 7 Configuration 5 Cryptography 1 Data Exposure 3 Data Validation 2 Component Breakdown Istio 10 Istio Sidecar 3 Istioctl 2 Pilot

百度内部基础设施层,服务发现。 ②bns-agent,服务发现接入层。 ㊟ 内核劫持：Loopback方案 Ø loopback地址的管理和分配。 Ø 需要打通业务和loopback之间的映射管 理。 RPC劫持：可扩展方案 Ø envoy启动后注册port到bns-agent。 Ø rpc框架查询bns-agent IP与治理策略数据。 Ø bns-agent判断否使用envoy进行服务治理。 。 Ø rpc框架根据反馈的IP，治理策略信息请求对 应IP，会cache数据，需要即时更新。 Ø envoy离线或者被干预则立即通知bns-agent， fallback会使用原有治理策略。 #IstioCon 架构介绍 Ø Mesh控制中心： ü 运维中心：基于Mesh的统一运维操作中心。 ü 配置中心：维护模块上下游拓扑，管理路由配 置、通信策略。 ü 上线中心：管理Mesh组件版本，统一上线入口。 优秀策略支持给业务方跨语言跨框架使用。 2. 支持LocalityAware Plus负载均衡策略，提 升单点容错能力。 业务价值 降低业务因Redis回退引发的雪崩问题。（业务层RPC 框架Retry策略托管到Mesh，通过平响分位值动态抑 制BP请求） Mesh价值 1. 业务无需代码改动即可开启，在线调整backup超时 分位值、熔断阈值。 2. 支持动态调整配置参数，对接智能调参系统。

lot-Agent——管理生命周期（PA） u启动envoy u热重启envoy u监控envoy u优雅关闭envoy启动envoy ü监听/etc/certs目录 ü生成envoy静态配置文件envoy-rev0.json ü通过exec.Command启动 envoy并监听状态 • 文件配置文档 • 启动参数文档热重启envoy热重启涉及以下步骤 • Pilot-Agent只是负责启动S，其他步骤由envoy完成。 Pilot-Agent只是负责启动S，其他步骤由envoy完成。 • 1. 启动另外一个S进程（Secondary process） • 2. S通知P（Primary process）关闭其管理的端口，由S接管 • 3. S加载配置，开始绑定listen sockets，在这期间使用UDS从P获取合适的listen sockets • 4. S初始化成功，通知P停止监听新的链接并优雅关闭未完成的工作 • 5. 在P优雅 oy ü获取非正常退出状态 ü抢救机制触发 ü抢救令牌减少一个（总共10个） ü在2(n-1) * 200毫秒后执行（为什么不立即执行） ü失败再次触发抢救机制 ü10个令牌用完，没有抢救成功，放弃退出优雅关闭envoy ü K8s发送SIGTERM信号让容器优雅关闭 ü Pilot-Agent接收信号通过context关闭子服务，发送SIGKILL关闭envoy ü Envoy不支

/pilot/cmd/pilot-discovery go run ./pilot/cmd/pilot-agent #IstioCon Fully Local go run ./pilot/cmd/pilot-discovery go run ./pilot/cmd/pilot-agent + Fast! Bottleneck is go compilation time + Trivial Cluster Remote Istiod, local proxy go run ./pilot/cmd/pilot-agent #IstioCon Cluster Remote Istiod, local proxy go run ./pilot/cmd/pilot-agent + Rapid iteration - Very different from production environment

destination: host: {revision-3}. 51ch62kjrnd.svc.cluster.local weight: 10 - destination: host: {revision-2}. 51ch62kjrnd.svc.cluster.local weight: 90 Knative Service Inspection #IstioCon - Security with Service provides aggregated data of Knative Service ready duration. o Knative Performance Testing Framework 2 Design #IstioCon o Ingress gateway MEM has linear growth, and it consumes ~=750k for 1 Knative Service Istio (1.7.3) istio- ingressgateway 2 vCPU 2 vCPU 2 Gi 4 Gi Y, min=3, max=20 Istiod 1 vCPU 1 vCPU 2 Gi 4 Gi Y, min=3, max=6 Knative Networking-istio 30m 80Mi 900m 2 Gi N #IstioCon Istio scalability