domain socket)，可以被 下游客户端连接。在 Envoy 中,Listener 可以绑定到端口上直接对外服务，也可以不绑 定到端口上，而是接收其他 listener 转发的请求。 • Cluster：集群是指 Envoy 连接的一组上游主机，集群中的主机是对等的，对外提供相 同的服务，组成了一个可以提供负载均衡和高可用的服务集群。Envoy 通过负载均衡 策略决定将请求路由到哪个集群成员。 协议的主要概念： • Listener Discovery Service (LDS) : 监听器发现服务。 • Route Discovery Service(RDS) : 路由发现服务。 • Cluster Discovery Service (CDS)： 集群发现服务。 • Endpoint Discovery Service (EDS) ：集群中的服务实例发现服务。 • Secret Discovery – 数据面 – Istio 中的 Envoy Sidecar 配置 Istio中的 Envoy Sidecar 配置： • Istio 通过 Listener、Route Config 和 Cluster 为 Mesh 中的 Envoy 生成了入向和出向两个不同方向的处理流程的配 置。 • 在 Envoy 的基础上增加了 VirtualInboundListener，VirtualOutboun

Technologies Co., Ltd. All rights reserved. Page 9 Envoy启动配置及xDS listener router upstream pool Envoy cluster istiod pilot-agent LDS RDS CDS EDS tls证书 管理 SDS CSR创建证书 stat tracing 支持采集或 主动上报 监控系统 过滤器 service.route.v3.RouteDiscoveryService/StreamRoutes CDS 上游cluster配置 POST /envoy.service.cluster.v3.ClusterDiscoveryService/StreamClusters EDS 上游cluster endpoint配置 POST /envoy.service.endpoint.v3.EndpointD • 启动admin RESTful监听，处理运行状态输出，prometheus收集等请求 • 定期将工作线程内监控数据stat进行合并 • 定期刷新DNS信息，加速域名解析。 • 目标cluster内主机列表健康状态判断。 • worker线程： • 通过启动配置参数concurrency指定，不支持动态调整。 • 启动virtualoutbound/virtualinbound网络

Virtual Machine Integration Odyssey, Jimmy Song #IstioCon V0.2 Mesh Expansion ● Prerequisites ○ IP connectivity to the endpoints in the mesh ○ Istio control plane services (Pilot, Mixer, CA) accessible Internal Load Balancers (ILBs) for Kube DNS, Pilot, Mixer and CA ○ Generate configs for VMs, incl. `cluster.env`, DNS config, Istio authN secrets etc. ○ Setup dnsmasq, Istio components in the VM and verify Dnsmasq accepts DNS queries 2. Access the built-in Kube DNS (exposed by ILB) 3. Obtain the Cluster IP resolved 4. Traffic intercepted by the sidecar proxy 5. xDS ■ Traffic forwarded to ingress in

Address Preserve #IstioCon What is the use case of original address 1. Sticky Session: based on ip hash, traffic from same client is forwarded to the same backend 2. Security Policy: set white/black Original Src Addr  L3 • LVS, one connection • HAProxy transparent mode, two connections  L4 • Add IP in TCP Protocol options • Proxy Protocol  L7 • HTTP header “x-forwarded-for” • User Protocol PREROUTING chain intercept packet and send it to INPUT ③ LVS work on INPUT, modify the packet dest ip + port and forward it to POSTROUTING ④ send out to real server Note: Only one connection between

○ HTTP JWT Auth ○ Redis Auth ○ ... IP Data IP Header TCP Data TCP Header Layer-7 Header Data #IstioCon What Do We Get From Istio? IP Data IP Header TCP Data TCP Header Layer-7 Management for non-HTTP/gRPC - only layer-3 to layer-6 ● Routing based on headers under layer-7 ○ IP address ○ TCP Port ○ SNI ● Observability - only TCP metrics ○ TCP sent/received bytes ○ TCP opened/closed Security ○ Connection level authentication: mTLS ○ Connection level authorization: Identity/Source IP/ Dest Port ○ Request level auth is impossible #IstioCon BookInfo Application - AwesomeRPC ProductPage

tenants. • Knative has knative-ingress-gateway for external access and knative-local-gateway for cluster local access. They use Istio gateway service istio-ingressgateway as its underlying service. Knative blue.51ch62kjrnd.svc.cluster.local http: route: - destination: host: {revision-3}. 51ch62kjrnd.svc.cluster.local weight: 10 - destination: host: {revision-2}. 51ch62kjrnd.svc.cluster.local weight: 90 Knative multiple shard k8s clusters, each cluster should support 1000 sequential (interval 5s) Knative service provisionings with route ready time <= 30s. Type Info K8s Cluster Capacity 12 nodes in 3 zones, 16

focus testing efforts. Istio does not currently have a reference design for what an ideal Kubernetes cluster with Istio running within it. Instead, NCC Group used various hosting options (i.e. Minikube, GKE controlPlaneSecuri ty configuration directive or other means. This left all default services exposed within the cluster. • The default istio profile that is labeled for produc- tion lacks many hardening controls and Pilot admin interface exposes unnecessary ser- vices and is accessible to anyone within a default cluster. • The Envoy Proxy admin port is exposed via the Istio sidecar and would allow a malicious workload

on-demand ● Passthrough mode during rollout ● Service entry to connect internal proxy ● Kubernetes Cluster-IP services deployed across clusters #IstioCon Rollout - Istio setup and Microservices ● Export Latency improvement #IstioCon Tooling and Automation ● Automate the Istio setup during Kubernetes cluster creation ● Automated endpoint config creation on new micro-service creation or updation ● Templatise gracefully (SIGINT, SIGTERM) ● Automate for easy management of setup across environments ● Ignore ports / IP as applicable - consul ● Namespace isolation helps reduce Istio proxy resources #IstioCon Next Steps

specific IP addresses to access our endpoints. Drawback: ● Not the preferred approach suggested from security team ● Maintenance a lot of endpoint for each GoPay partner with specific IP seems burden Security concern about internal attacks (we don’t know who are using those IP, only service that communicate with us or it’s NAT IP that used by all services) Implementing Mutual TLS Centralized Certificate certificate lifecycle for HTTPS and mutual TLS communication. ● Renew & sync to our Kubernetes cluster, also support syncing to VM with an agent installed, this is also used by our partners as well

workload instance. kubernetes://redis- master-2353460263- 1ecey.my-namespace source.ip ip_address Source workload instance IP address. 10.0.0.117 source.labels map[string, string] A map of key-value pairs attached to the source instance. version => v1 destination.port int64 The recipient port on the server IP address. 8080 request.time timestamp The timestamp when the destination receives the request. This "bytes":{"0":"rBQDuw==","150":"AAAAAAAAAAAAAP//rBQDqg=="} üreq.DefaultWords ： • ["istio-pilot.istio-system.svc.cluster.local", • "kubernetes://istio-pilot-8696f764dd-fqxtg.istio-system", • "3a7a649f-4eeb-4d70-972c-ad2d43a680af"