North-South Load Balancing of Kubernetes Services with eBPF/XDP Martynas Pumputis (Isovalent) October 28, 2020 10.0.0.1 10.0.0.2 10.0.0.3 httpd httpd “httpd” service 10.0.0.1:30000 10.0.0.2:30000 KUBE-FORWARD -m comment --comment "kubernetes forwarding rules" -m mark --mark 0x4000/0x4000 -j ACCEPT -A KUBE-FORWARD -s 10.217.0.0/16 -m comment --comment "kubernetes forwarding conntrack pod source rule" --ctstate RELATED,ESTABLISHED -j ACCEPT -A KUBE-FORWARD -d 10.217.0.0/16 -m comment --comment "kubernetes forwarding conntrack pod destination rule" -m conntrack --ctstate RELATED,ESTABLISHED -j ACCEPT

Concepts Component Overview Terminology Networking Network Security eBPF Datapath Observability Kubernetes Integration Multi-Cluster (Cluster Mesh) Getting Help FAQ Slack GitHub Training Enterprise support Matrix Linux Kernel Required Kernel Versions for Advanced Features Key-Value store clang+LLVM iproute2 Firewall Rules Mounted eBPF filesystem Privileges Upgrade Guide Running pre-flight check (Required) Layer 4 Examples Layer 7 Examples Deny Policies Host Policies Layer 7 Protocol Visibility Using Kubernetes constructs in policy Endpoint Lifecycle Troubleshooting Monitoring & Metrics Cilium Metrics Hubble

Concepts Component Overview Terminology Networking Network Security eBPF Datapath Observability Kubernetes Integration Multi-Cluster (Cluster Mesh) Getting Help FAQ Slack GitHub Training Enterprise support Considerations Linux Kernel Required Kernel Versions for Advanced Features Key-Value store clang+LLVM iproute2 Firewall Rules Mounted eBPF filesystem Privileges Upgrade Guide Running pre-flight check (Required) Layer 4 Examples Layer 7 Examples Deny Policies Host Policies Layer 7 Protocol Visibility Using Kubernetes constructs in policy Endpoint Lifecycle Troubleshooting Monitoring & Metrics Cilium Metrics Hubble

Orchestrators Concepts Component Overview Terminology Networking Network Security eBPF Datapath Kubernetes Integration Multi-Cluster (Cluster Mesh) Getting Help FAQ Slack GitHub Security Bugs Operations Matrix Linux Kernel Required Kernel Versions for Advanced Features Key-Value store clang+LLVM iproute2 Firewall Rules Mounted eBPF filesystem Privileges Upgrade Guide Running pre-flight check (Required) Layer 3 Examples Layer 4 Examples Layer 7 Examples Host Policies Layer 7 Protocol Visibility Using Kubernetes constructs in policy Endpoint Lifecycle Troubleshooting Monitoring & Metrics Cilium Metrics

Concepts Component Overview Terminology Networking Network Security eBPF Datapath Observability Kubernetes Integration Multi-Cluster (Cluster Mesh) Getting Help FAQ Slack GitHub Training Enterprise support Considerations Linux Kernel Required Kernel Versions for Advanced Features Key-Value store clang+LLVM iproute2 Firewall Rules Mounted eBPF filesystem Privileges Upgrade Guide Running pre-flight check (Required) Layer 4 Examples Layer 7 Examples Deny Policies Host Policies Layer 7 Protocol Visibility Using Kubernetes constructs in policy Endpoint Lifecycle Troubleshooting Monitoring & Metrics Cilium Metrics Hubble

the Cilium architecture and how these components integrate with existing architectures, such as Kubernetes. Installation : Details instructions for installing, configuring, and troubleshooting Cilium in Datapath Scale Kubernetes Integration Getting Help FAQ Slack GitHub Security Bugs Integrations Kubernetes Introduction Concepts Requirements Configuration Network Policy Endpoint CRD Kubernetes Compatibility Matrix Linux Kernel Advanced Features and Required Kernel Version Key-Value store clang+LLVM iproute2 Firewall Rules Privileges Upgrade Guide Running pre-flight check (Required) Upgrading Cilium Step

the Cilium architecture and how these components integrate with existing architectures, such as Kubernetes. Installation : Details instructions for installing, configuring, and troubleshooting Cilium in Datapath Scale Kubernetes Integration Getting Help FAQ Slack GitHub Security Bugs Integrations Kubernetes Introduction Concepts Requirements Configuration Network Policy Endpoint CRD Kubernetes Compatibility Matrix Linux Kernel Advanced Features and Required Kernel Version Key-Value store clang+LLVM iproute2 Firewall Rules Privileges Upgrade Guide Running pre-flight check (Required) Upgrading Micro Versions

the Cilium architecture and how these components integrate with exis�ng architectures, such as Kubernetes. Installa�on : Details instruc�ons for installing, configuring, and troubleshoo�ng Cilium in different Datapath Scale Kubernetes Integra�on Ge�ng Help FAQ Slack GitHub Security Bugs Integra�ons Kubernetes Introduc�on Concepts Requirements Configura�on Network Policy Endpoint CRD Kubernetes Compa�bility Troubleshoo�ng Requirements Summary Linux Distribu�on Compa�bility Matrix Linux Kernel Key-Value store clang+LLVM iproute2 Firewall Rules Privileges Upgrade Guide Running a pre-flight DaemonSet Upgrading Micro Versions Upgrading

挑战1：微服务、多语言、多协议环境下，端到端观测 复杂度上升，埋点成本居高不下 Kubernetes 容器 网络、操作系统、硬件 基础设施层复杂度日益增加 如何关联? 挑战3：数据散落，工具多， 缺少上下文，排查效率低下 业务应用 应用框架 容器虚拟化 系统调用 内核 应用性能监控（APM） Kubernetes监控 Kubernetes组件异常： Scheduler, KCM， etcd，api-server 内核异常：进程调度， 内存管理，文件管理， 夯机宕机，资源异 常… 应用组件异常：线程池满，数据库连接无法获取， OOM，文件读取错误… 无法自顶向下端到端 串联导致棘手问题频 发。 Kubernetes下的可观测 Golang + eBPF实现数据采 集 第四部分 eBPF在可观测领域的优势 无侵入 多语言/多协议/多框架 全栈覆盖 无侵入性 • 无需修改代码 • 无需重启应用 Error()) } links = append(links, pl) return nil } bpf2go 01. 副标题 //go:generate go run github.com/cilium/ebpf/cmd/bpf2go -cc clang -cflags $BPF_CFLAGS -type insp_pl_event_t - type insp_pl_metric_t

Managed Kubernetes to App Platform: 1.5 Years of Cilium Usage at DigitalOcean Timo Reimann, DigitalOcean October 28, 2020 digitalocean.com History / Context ● DigitalOcean Kubernetes Service aka aka DOKS: our managed Kubernetes offering ● Started out using Flannel but decided to move to Cilium in late 2018 for a couple of reasons: ○ support for NetworkPolicies ○ feature-rich CNI implementation ● cilium-agent managed as DaemonSet on each worker node ● cilium-operator managed as Deployment (2 replicas / HA mode in latest releases) on workers ● cilium-agent running on control plane to enable