Istio Security Assessment Google August 6, 2020 – Version 1.1 Prepared for Arun Kumar R Prepared by Mark Manning Jeff Dileo Divya Natesan Andy Olsen Feedback on this project? https://my.nccgroup Synopsis In the summer of 2020, Google enlisted NCC Group to perform an assessment on the open-source version of Istio and all of its components. Istio is a modern service mesh technology stack often used within TLS certificates, provides workload identity, and includes a builtin authorization system facilitated by its control plane. The goal of the assessment was to identify security issues related to the Istio

Shortcoming 1: Controlling the running order for containers Stabilizing Istio Kubernetes lacks good control APIs to customize the containers lifecycle in a pod. There is no official way to instruct a pod Injecting sidecars, HTTP/2 LoadBalancing ● Traffic shifting for canaries Build confidence in the system and understanding of Istio. Then you can onboard some users, get feedback, improve, rinse and repeat said 32 A full mesh is utopian, know what you need only Stabilizing Istio The reality: ● The control plane is burning down when pushing your thousand services updates to the hundreds of proxies running

Golang 1 vulnerability found that affected Googles managed Istio offering 11 issues found ● 5 system resource exhaustion ● 1 arbitrary file write ● 1 missing file close ● 1 certificate skipping offers more advanced features to support A/B testing, canary deployments, rate limiting, access control, encryption and end-to-end authentication. Istio itself is implemented in Go which shields the project consist of Envoy proxies and an Istio-agent and manage network traffic between microservices. The control plane is responsible for applying user configuration to the proxies. The following diagram demonstrates

Ingress Gateway. • Enable Secret Discovery Service (SDS) to monitor and mount secrets under istio-system to ingress gateway which contains credentials for https support of multi tenants. • Knative has application traffic end to end in production • Allow platform to use Istio authorization policy to control the access to each Knative service based on Istio service roles. How Istio is leveraged in a Knative Info K8s Cluster Capacity 12 nodes in 3 zones, 16 vCPU * 64 Gi MEM Knative Version Knative 0.16, 0.17, 0.18 Istio Version 1.5, 1.6, 1.7 Istio scalability optimization during Knative Service provisioning

#IstioCon V0.2 Mesh Expansion ● Prerequisites ○ IP connectivity to the endpoints in the mesh ○ Istio control plane services (Pilot, Mixer, CA) accessible from the VMs ○ (optional) Kubernetes DNS server accessible DNS query httpbin.ns1.svc.cluster.local 2. Cached DNS response – 10.4.4.4 DNS queries to the system configured name servers. Envoy does not use the agent’s DNS cache. http req to 10.4.4.4 GET /status/200 http req to 172.16.1.3 GET /status/200 #IstioCon V1.8 Smart DNS Proxy: A Step Further ● Taking control of DNS! ○ VMs to Kubernetes integration ○ Reduced load on your DNS servers w/ faster resolution

values-overrides.yaml Install using helm install istiod manifests/charts/istio-control/istio-discovery \ -n istio-system --values values-overrides.yaml meshConfig: defaultConfig: proxyMetadata: inlineBytes' | \ sed 's/"//g' | base64 --decode | openssl x509 -noout -text Certificate: Data: Version: 3 (0x2) Serial Number: … Signature Algorithm: sha256WithRSAEncryption

Hierarchy of control planes ● Global Control Plane ○ Users provide application specs to Global Control-Plane ○ Syncs specs to AZ control-planes ○ Hosts global services - Global IPAM, Access-control Policy Policy store, etc. ● AZ Control Plane ○ Syncs specs to workload K8s clusters in the AZ ○ Shared-Nothing Architecture ■ Hosts services catering to the AZ, e.g., AZ IPAM, Network Load-balancers, etc K8s Cluster K8s Cluster K8s Cluster K8s Cluster AZ Control Plane AZ Control Plane AZ Control Plane Global Control Plane Region Rn Delegate #IstioCon Load balancing & Traffic

Different namespace • Project runs as tenant, need control rights Solution cluster connect core cluster with Istio multi-cluster - Replicated control planes Some standalone cluster without Istio can #IstioCon Secure Platform – Authorization Policy Using Authorization Policy enables access control on workloads in the mesh. For request from ingressgateway, need verify token For request from same Platform – Extra Authorization Version 1 : Istio Mixer authz adapt Implement role-based authorization – whether this user can access this api based on its role => Version 2: Envoyfilter ext_authz #IstioCon

with NIST ● Author SP 800-204 series on microservice security ● R&D on Next Generation Access Control (NGAC) ● Exclusively co-host annual zero trust multi-cloud conference Best in Class Team ● Creators operating an application Why is Istio? TSB: The Application-Aware Networking Platform Istio: Control Plane Tetrate Service Bridge: Management Plane Envoy: Data Plane Workload (Service) POD Workload availability & resiliency enabling active-active deployments ● Cross cluster security policies & access control ● Unified telemetry and availability reporting ● Service discovery across multiple clusters ●

traffic ● Authorize in mesh traffic ● Summary #IstioCon Istio Architecture Connect, secure, control, and observe services. #IstioCon Security Architecture #IstioCon Bookinfo architecture without istio.io/v1beta1“ kind: "RequestAuthentication“ metadata: name: "jwt-example“ namespace: istio-system spec: selector: matchLabels: istio: ingressgateway jwtRules: - issuer: testing@secure security.istio.io/v1beta1 kind: AuthorizationPolicy metadata: name: require-jwt namespace: istio-system spec: action: ALLOW rules: - from: - source: requestPrincipals: ["testing@secure