#IstioCon Using ECC Workload Certificates (pilot-agent environmental variables) Jacob Delgado / Aspen Mesh #IstioCon ECC workload certificates ● In various environments, the need for x509 certificates cryptography (using ECDSA P-256) to use this feature ● Only ECDSA P-256 is supported #IstioCon pilot-agent environmental variables Disclaimer: Environmental variables and their use are considered experimental set the ECC_SIGNATURE_ALGORITHM environmental variable on sidecar ejection to ECDSA for use by pilot-agent ○ For gateways this environmental variable also must be set on installation/upgrade #IstioCon

tecture of Istio as it is deployed within common environments such as Kubernetes clusters. • Istio Pilot: The service running within the istiod service that handles service discovery. • Istio Ingress/Egress: lacks many hardening controls and should be replaced with a more secure-by-default option. • The Pilot admin interface exposes unnecessary ser- vices and is accessible to anyone within a default cluster Data Exposure 3 Data Validation 2 Component Breakdown Istio 10 Istio Sidecar 3 Istioctl 2 Pilot 3 Key Critical High Medium Low Informational 3 | Google Istio Security Assessment Google / NCC

#IstioCon Fully Local go run ./pilot/cmd/pilot-discovery go run ./pilot/cmd/pilot-agent #IstioCon Fully Local go run ./pilot/cmd/pilot-discovery go run ./pilot/cmd/pilot-agent + Fast! Bottleneck is #IstioCon Cluster Remote Istiod, local proxy go run ./pilot/cmd/pilot-agent #IstioCon Cluster Remote Istiod, local proxy go run ./pilot/cmd/pilot-agent + Rapid iteration - Very different from production #IstioCon Local Istiod, remote proxy Cluster go run ./pilot/cmd/pilot-discovery #IstioCon Local Istiod, remote proxy Cluster go run ./pilot/cmd/pilot-discovery + All of the benefits of running Istiod

可观察性：遥测数据、调用跟踪、服务拓扑 通信安全： 服务身份认证、访问鉴权、通信加密 Proxy Application Layer Service 1 Istio 流量管理 – 概览 • 控制面下发流量规则： Pilot • 数据面标准协议：xDS • 集群内Pod流量出入： Sidecar Proxy • 集群外部流量入口：Ingress Gateway • 集群外部流量出口：Egress Gateway（可选 balancing • Time out • Retries • Circuit breaker • Routing • Auth • Telemetry collecting 外部流量出口 外部流量入口 Pilot 2 Istio 流量管理 – 控制面 两类数据： q 服务数据（Mesh 中有哪些服务？缺省路由） v Service Registry § Kubernetes：原生支持 § Consul、Eureka Istio 流量管理 – 控制面 – 服务发现 • K8s Service ： Pilot 直接支持 • ServiceEntry： 手动添加 Service 到 Pilot 内部注册表中 • WorkloadEntry：单独添加 Workload，对于虚机支持更友好 • MCP 适配器： 将第三方注册表中的服务加入到 Pilot 中 Consul MCP Adapter https://github.

Copyright © Huawei Technologies Co., Ltd. All rights reserved. Page 6 Envoy原理及总体架构-启动 istiod Pilot-agent Pilot-agent apiServer iptables iptables Envoy client backend 8123 Virtual outbound -15001 Envoy les规则 • istio-proxy容器启动pilot-agent进程，使用UID=1337 GID=1337创建Envoy启动命令行与配置文件 • 可以通过自定义deployment内istio注解sidecar.istio.io/inject: “false”跳过自动注入过程，或修改部分启动参数。 • 2. 控制面通信 • Pilot-agent进程本身创建UDS接收Envoy连接， 用于证书更新下发。并且与istiod建立证书更新通道。 • Envoy 通过pilot-agent转发机制与istiod建立长连接，通过xDS协议接收系统下发的监听器、路由、集群节点等更新信息。 • 3. 数据面通信 • 客户端请求进入容器网络，并被iptables规则拦截，经过DNAT后进入Envoy virtualOutbound监听器 • virtualOutbound经过监听过滤器

o Istiod MEM bumped with large numbers of Knative Services (#25532) Mem usage optimization of pilot resolved this issue. • Tune CPU/MEM to ensure enough capacity Leveraged Metrics to monitor Istio Istiod. o From envoy logs, transient 503 UH "no healthy upstream" errors. o From Grafana dashboard, Pilot Pushes shows long latencies. • Detect and analyze Istio scalability issue #IstioCon o Radom peaks Knative Service provisioning o PILOT_ENABLE_EDS_DEBOUNCE=true,PILOT_DEBOUNCE_AFTER=100ms and PILOT_DEBOUNCE_MAX=10s are the env vars on pilot that can be tuned. o Set PILOT_DEBOUNCE_AFTER=1s helps under

sidecar Envoys ○ Measure Config convergence time ■ Time taken by all sidecars to get config from Pilot without any errors ■ For thousands of services & endpoints ■ With different churn rates of Pods time from single Pilot instance to 0 - 3,000 sidecars < 1 second ○ Pilot CPU & memory within acceptable limits: < 10 cores, 25 GB memory ○ Pilot can scale horizontally ● Need to tune PILOT_DEBOUNCE_AFTER PILOT_DEBOUNCE_AFTER, PILOT_DEBOUNCE_MAX, PILOT_PUSH_THROTTLE, etc. params of Istio Pilot #IstioCon Future Direction ● Support for on-demand config pushes to Envoy via Incremental XDS ● Support for multiple trust

Service Mesh 改造 探索 Istio 技术点 Dubbo 场景下 的改造 • 对比传统微服务架构 • 和 Service Mesh 化 之后有哪些优缺点 • MCP • Pilot • xDS • MOSN 结合 Istio 的技术点， 介绍多点生活目前的 探 索 以 及 服 务 发 现 Demo 的演示3/23 为什么需要 Service Mesh 改造 /01 Source 和 Sink： • Source 是资源提供方（server），资源变化了推送给订阅者（Pilot），Istio 1.5 之前这个 角色就是 Galley 或者自定义 MCP Server； • Sink 资源的订阅者（client），在 Istio1.5 之前这个角色就是 Pilot 和 Mixer，都是订阅 Galley 或者自定义 MCP Server 的资源；8/23 MCP mcpserver demo: https://github.com/champly/mcpserver9/23 Pilot Pilot 负责网格中的流量管理以及控制面和数据面之间的配置下发，在 Istio1.5 之后合并了 Galley、Citadel、Sidecar-Inject 和 Pilot 成为 Istiod。 功能 • 根据不同的平台（Kubernetes、Console） 获取一些资源（Kubernetes

觉。目录Pilot-Agent——管理生命周期（PA） u启动envoy u热重启envoy u监控envoy u优雅关闭envoy启动envoy ü监听/etc/certs目录 ü生成envoy静态配置文件envoy-rev0.json ü通过exec.Command启动 envoy并监听状态 • 文件配置文档 • 启动参数文档热重启envoy热重启涉及以下步骤 • Pilot-Agen ü10个令牌用完，没有抢救成功，放弃退出优雅关闭envoy ü K8s发送SIGTERM信号让容器优雅关闭 ü Pilot-Agent接收信号通过context关闭子服务，发送SIGKILL关闭envoy ü Envoy不支持优雅关闭，需要通过金丝雀或蓝绿部署方式实现 Envoy优雅关闭实现方式讨论：#3307 #2920Pilot-Discovery——配置中心（PD） uv1版本和v2版本之间的区别 u建立缓存配置 ,"150":"AAAAAAAAAAAAAP//rBQDqg=="} üreq.DefaultWords ： • ["istio-pilot.istio-system.svc.cluster.local", • "kubernetes://istio-pilot-8696f764dd-fqxtg.istio-system", • "3a7a649f-4eeb-4d70-972c-ad2d43a680af"

AwesomeRPC in Istio? #IstioCon How to Manage AwesomeRPC Traffic in Istio? Pilot Envoy Code changes at the Pilot side: ● Add AwesomeRPC support in VirtualService API ● Generate LDS/RDS for Reviews v1 Pilot EnvoyFilter ● Match: TCP Proxy in the listener filterChain ● Operation: Replace with an AwesomeRPC filter XXXX XXXX XXXX Pilot replaces the TCP proxy in the Istio configuration CRD, by which we can apply a “patch” to the Envoy configuration generated by Pilot. #IstioCon EnvoyFilter Example - Dubbo Traffic Splitting Replace TCP proxy in the outbound listener