North-South Load Balancing of Kubernetes Services with eBPF/XDP Martynas Pumputis (Isovalent) October 28, 2020 10.0.0.1 10.0.0.2 10.0.0.3 httpd httpd “httpd” service 10.0.0.1:30000 10.0.0.2:30000 FORWARD filter FORWARD mangle POSTROUTING nat POSTROUTING TC egress host httpd pod lxc0 eth0 XDP httpd httpd 1010101010111 1010101010111 1010101010111 DSR httpd httpd 1010101010111 1010101010111

how to troubleshoot Cilium in different deployment modes. BPF and XDP Reference Guide : Provides a technical deep dive of BPF and XDP technology, primarily focused at developers. API Reference : Details Container Images Documenta�on CI / Jenkins Release Management Developer’s Cer�ficate of Origin BPF and XDP Reference Guide BPF Architecture Toolchain Program Types Further Reading API Reference Introduc�on descrip�on. For a more thorough documenta�on on specifics of each hook see BPF and XDP Reference Guide. XDP: The XDP BPF hook is at the earliest point possible in the networking driver and triggers a

how to troubleshoot Cilium in different deployment modes. BPF and XDP Reference Guide : Provides a technical deep dive of BPF and XDP technology, primarily focused at developers. API Reference : Details Pull-Request Builds With Jenkins Using Jenkins for testing CI Failure Triage Infrastructure details BPF and XDP Reference Guide BPF Architecture Toolchain Program Types Further Reading API Reference Introduction description. For a more thorough documentation on specifics of each hook see BPF and XDP Reference Guide. XDP: The XDP BPF hook is at the earliest point possible in the networking driver and triggers a

how to troubleshoot Cilium in different deployment modes. BPF and XDP Reference Guide : Provides a technical deep dive of BPF and XDP technology, primarily focused at developers. API Reference : Details individual packages Running individual tests Automatically run unit tests on code changes BPF and XDP Reference Guide BPF Architecture Instruction Set Helper Functions Maps Object Pinning Tail Calls Introspection Tracing pipe Miscellaneous Program Types XDP tc (traffic control) Further Reading Kernel Developer FAQ Projects using BPF XDP Newbies BPF Newsletter Podcasts Blog posts Books Talks Further

how to troubleshoot Cilium in different deployment modes. BPF and XDP Reference Guide : Provides a technical deep dive of BPF and XDP technology, primarily focused at developers. API Reference : Details individual packages Running individual tests Automatically run unit tests on code changes BPF and XDP Reference Guide BPF Architecture Instruction Set Helper Functions Maps Object Pinning Tail Calls Introspection Tracing pipe Miscellaneous Program Types XDP tc (traffic control) Further Reading Kernel Developer FAQ Projects using BPF XDP Newbies BPF Newsletter Podcasts Blog posts Books Talks Further

how to troubleshoot Cilium in different deployment modes. BPF and XDP Reference Guide : Provides a technical deep dive of eBPF and XDP technology, primarily focused at developers. API Reference : Details individual packages Running individual tests Automatically run unit tests on code changes BPF and XDP Reference Guide BPF Architecture Instruction Set Helper Functions Maps Object Pinning Tail Calls Introspection Tracing pipe Miscellaneous Program Types XDP tc (traffic control) Further Reading Kernel Developer FAQ Projects using BPF XDP Newbies BPF Newsletter Podcasts Blog posts Books Talks Further

how to troubleshoot Cilium in different deployment modes. BPF and XDP Reference Guide : Provides a technical deep dive of eBPF and XDP technology, primarily focused at developers. API Reference : Details individual packages Running individual tests Automatically run unit tests on code changes BPF and XDP Reference Guide BPF Architecture Instruction Set Helper Functions Maps Object Pinning Tail Calls Introspection Tracing pipe Miscellaneous Program Types XDP tc (traffic control) Further Reading Kernel Developer FAQ Projects using BPF XDP Newbies BPF Newsletter Podcasts Blog posts Books Talks Further

how to troubleshoot Cilium in different deployment modes. BPF and XDP Reference Guide : Provides a technical deep dive of eBPF and XDP technology, primarily focused at developers. API Reference : Details individual packages Running individual tests Automatically run unit tests on code changes BPF and XDP Reference Guide BPF Architecture Instruction Set Helper Functions Maps Object Pinning Tail Calls Introspection Tracing pipe Miscellaneous Program Types XDP tc (traffic control) Further Reading Kernel Developer FAQ Projects using BPF XDP Newbies BPF Newsletter Podcasts Blog posts Books Talks Further

linux 5.14 版本，eBPF 有32种类型程序。而 cilium 主要使用了如下类型程序： • sched_cls 。cilium在内核 TC 处实现数据包转发、负载均衡、过滤 • xdp 。cilium在内核 XDP 处实现数据包的转发、负载均衡、过滤 • cgroup_sock_addr 。cilium在 cgroup 中实现对service解析 • sock_ops + sk_msg。记录本 iptables 等 技术，降低了访 问延时。例如在相同环境下，service 数量达到3K，kube-proxy iptables下 的的延时为0.6ms，而cilium的延时为 0.3ms XDP 加速南北向 nodePort 访问 cilium 借助 eBPF 程序 ，能快速完 成 nodePort 、 LoadBalancer service 的解析和转发，其转发性能能比肩 DPDK 压力下，节省了30%的CPU利用率 2. XDP的性能上限极高，可能是 TC 的 10 倍左右 raw PREROUTING mangle PREROUTING nat PREROUTING tc ingress conntrack filter FORWARD mangle POSTROUING nat POSTROUING tc egress routing XDP kernel

and XDP Based L4 Load Balancer Service • Part of our private cloud service since 2017 • 5100 private, 760 public VIPs • k8s CCM integration (Type: LoadBalancer) L4LB Node L4LB Architecture XDP DPlane acceleration using XDP (jp) • https://engineering.linecorp.com/ja/blog/intern2019- report-infra/ • https://www.janog.gr.jp/meeting/janog45/application/f iles/3815/7952/0335/009_srv6xdp_saito.pdf • UDP