Scaling a Multi-Tenant k8s Cluster in a Telco Pablo Moncada October 28, 2020 About MasMovil group ● 4th telecom company in Spain ● Provides voice and broadband services to +12M customers ● Several Services +3k CPU +2k Mem +5TB Nodes +300 kube-proxy replacement NetworkPolicy logging Multi-cluster DNS Aware NetworkPolicy Increased Istio security External Services TLS visibility Performance

technical deep dive of eBPF and XDP technology, primarily focused at developers. API Reference : Details the Cilium agent API for interacting with a local Cilium instance. Development Guide : Gives background Advanced Networking Cluster Mesh Operations Istio Concepts Component Overview Terminology Networking Network Security eBPF Datapath Observability Kubernetes Integration Multi-Cluster (Cluster Mesh) Getting Troubleshooting Component & Cluster Health Observing Flows with Hubble Observing flows with Hubble Relay Connectivity Problems Policy Troubleshooting etcd (kvstore) Cluster Mesh Troubleshooting Symptom

technical deep dive of eBPF and XDP technology, primarily focused at developers. API Reference : Details the Cilium agent API for interacting with a local Cilium instance. Development Guide : Gives background Terminology Networking Network Security eBPF Datapath Observability Kubernetes Integration Multi-Cluster (Cluster Mesh) Getting Help FAQ Slack GitHub Training Enterprise support Security Bugs Operations Scalability report Performance Evaluation Setup Evaluation Results Tuning Troubleshooting Component & Cluster Health Observing Flows with Hubble Observing flows with Hubble Relay Connectivity Problems Policy

technical deep dive of eBPF and XDP technology, primarily focused at developers. API Reference : Details the Cilium agent API for interacting with a local Cilium instance. Development Guide : Gives background Advanced Networking Cluster Mesh Operations Istio Concepts Component Overview Terminology Networking Network Security eBPF Datapath Observability Kubernetes Integration Multi-Cluster (Cluster Mesh) Getting Troubleshooting Component & Cluster Health Observing Flows with Hubble Observing flows with Hubble Relay Connectivity Problems Policy Troubleshooting etcd (kvstore) Cluster Mesh Troubleshooting Symptom

technical deep dive of BPF and XDP technology, primarily focused at developers. API Reference : Details the Cilium agent API for interacting with a local Cilium instance. Development Guide : Gives background Overview Terminology Networking Network Security eBPF Datapath Kubernetes Integration Multi-Cluster (Cluster Mesh) Getting Help FAQ Slack GitHub Security Bugs Operations System Requirements Summary Scalability report Performance Evaluation Setup Evaluation Results Tuning Troubleshooting Component & Cluster Health Observing Flows with Hubble Observing flows with Hubble Relay Connectivity Problems Policy

technical deep dive of BPF and XDP technology, primarily focused at developers. API Reference : Details the Cilium agent API for interacting with a local Cilium instance. Getting Started : Gives background 4 Examples Layer 7 Examples Kubernetes Endpoint Lifecycle Troubleshooting L7 Protocol Visibility API Rate Limiting Default Rate Limits Configuration Automatic Adjustment Metrics Understanding the log Agent Monitoring & Metrics Installation cilium-agent cilium-operator Troubleshooting Component & Cluster Health Connectivity Problems Policy Troubleshooting Symptom Library Useful Scripts Reporting a problem

technical deep dive of BPF and XDP technology, primarily focused at developers. API Reference : Details the Cilium agent API for interacting with a local Cilium instance. Developer / Contributor Guide : Troubleshooting Monitoring & Metrics Installation cilium-agent cilium-operator Troubleshooting Component & Cluster Health Connectivity Problems Policy Troubleshooting Symptom Library Useful Scripts Reporting a problem Guide BPF Architecture Toolchain Program Types Further Reading API Reference Introduction How to access the API Compatibility Guarantees API Reference Reference Command Cheatsheet Command utilities:

technical deep dive of BPF and XDP technology, primarily focused at developers. API Reference : Details the Cilium agent API for interac�ng with a local Cilium instance. Developer / Contributor Guide : Exported Metrics Cilium as a Kubernetes pod Cilium as a host-agent on a node Troubleshoo�ng Component & Cluster Health Connec�vity Problems Policy Troubleshoo�ng Automa�c Diagnosis Symptom Library Useful Scripts Guide BPF Architecture Toolchain Program Types Further Reading API Reference Introduc�on How to access the API Compa�bility Guarantees API Reference Reference Command Cheatsheet Command u�li�es: Command

shared cluster etcd Cilium in the DOKS Architecture Data Plane Node #1 cilium-agent Node #1 cilium-agent cilium-operator Node #1 cilium-agent cilium-operator Control Plane kube-api-server pretty smooth ○ moved from Cilium 1.4 initially to 1.8 today ○ retain old RBAC rules across certain cluster upgrades to avoid disruptions ● (Health checking) tooling really helpful in troubleshooting issues

policies ❏ Tools to help with network troubleshooting and policies ❏ Additional features like IPSec, Cluster Mesh, and more 12 Reduced iptables Complexity 13 CiliumNetworkPolicies Layer 7 HTTP Filtering