Hubble can answer questions such as: Service dependencies & communication map What services are communicating with each other? How frequently? What does the service dependency graph look like? What HTTP HTTP calls are being made? What Kafka topics does a service consume from or produce to? Network monitoring & alerting Is any network communication failing? Why is communication failing? Is it DNS? Is it requests? Application monitoring What is the rate of 5xx or 4xx HTTP response codes for a particular service or across all clusters? What is the 95th and 99th percentile latency between HTTP requests and responses

Hubble can answer questions such as: Service dependencies & communication map What services are communicating with each other? How frequently? What does the service dependency graph look like? What HTTP HTTP calls are being made? What Kafka topics does a service consume from or produce to? Network monitoring & alerting Is any network communication failing? Why is communication failing? Is it DNS? Is it requests? Application monitoring What is the rate of 5xx or 4xx HTTP response codes for a particular service or across all clusters? What is the 95th and 99th percentile latency between HTTP requests and responses

Hubble can answer questions such as: Service dependencies & communication map What services are communicating with each other? How frequently? What does the service dependency graph look like? What HTTP HTTP calls are being made? What Kafka topics does a service consume from or produce to? Network monitoring & alerting Is any network communication failing? Why is communication failing? Is it DNS? Is it requests? Application monitoring What is the rate of 5xx or 4xx HTTP response codes for a particular service or across all clusters? What is the 95th and 99th percentile latency between HTTP requests and responses

Hubble can answer questions such as: Service dependencies & communication map What services are communicating with each other? How frequently? What does the service dependency graph look like? What HTTP HTTP calls are being made? What Kafka topics does a service consume from or produce to? Network monitoring & alerting Is any network communication failing? Why is communication failing? Is it DNS? Is it requests? Application monitoring What is the rate of 5xx or 4xx HTTP response codes for a particular service or across all clusters? What is the 95th and 99th percentile latency between HTTP requests and responses

configuration. Why Cilium? The development of modern datacenter applications has shifted to a service-oriented architecture often referred to as microservices, wherein a large application is split into to transparently insert security visibility + enforcement, but does so in a way that is based on service / pod / container identity (in contrast to IP address identification in traditional systems) and requests with method GET and path /public/.*. Deny all other requests. Allow service1 to produce on Kafka topic topic1 and service2 to consume on topic1. Reject all other Kafka messages. Require the HTTP header

Reimann, DigitalOcean October 28, 2020 digitalocean.com History / Context ● DigitalOcean Kubernetes Service aka DOKS: our managed Kubernetes offering ● Started out using Flannel but decided to move to Cilium worker node ● cilium-operator managed as Deployment (2 replicas / HA mode in latest releases) on workers ● cilium-agent running on control plane to enable control/data plane connectivity ● Cilium state-keeping

configuration. Why Cilium? The development of modern datacenter applications has shifted to a service-oriented architecture often referred to as microservices, wherein a large application is split into to transparently insert security visibility + enforcement, but does so in a way that is based on service / pod / container identity (in contrast to IP address identification in traditional systems) and requests with method GET and path /public/.*. Deny all other requests. Allow service1 to produce on Kafka topic topic1 and service2 to consume on topic1. Reject all other Kafka messages. Require the HTTP header

container configura�on. Why Cilium? The development of modern datacenter applica�ons has shi�ed to a service- oriented architecture o�en referred to as microservices, wherein a large applica�on is split into to transparently insert security visibility + enforcement, but does so in a way that is based on service / pod / container iden�ty (in contrast to IP address iden�fica�on in tradi�onal systems) and can requests with method GET and path /public/.* . Deny all other requests. Allow service1 to produce on Ka�a topic topic1 and service2 to consume on topic1 . Reject all other Ka�a messages. Require the HTTP

TC 处实现数据包转发、负载均衡、过滤 • xdp 。cilium在内核 XDP 处实现数据包的转发、负载均衡、过滤 • cgroup_sock_addr 。cilium在 cgroup 中实现对service解析 • sock_ops + sk_msg。记录本地应用之间通信的socket，实现本地数据包的加速转发 加速同节点pod间通信 cilium 使用 eBPF 程序，借助 bpf_redirect() stack netfilter 加速东西向 nodePort 访问 �������������������� ������� request to nodeport 32000 of service pod3 worker node1 10.6.0.10 ������ ������������� ���������������������� worker node 3 10 10:10000 cgroup ebpf service DNAT connect sendmsg recvmsg getpeername bind cilium的Host-Reachable 技术，利 用eBPF程序，拦截应用在内核connect 、 sendmsg、 recvmsg 、getpeername 、 bind等系统调用，实现 service 的地址解 析，并且伪装通信目的地址，让上层应用

Linux kernel, ... ● Contributor to Linux kernel networking & BPF subsystems Goal Run a TCP echo service on ports 7, 77, and 777 … using one TCP listening socket. Fun? We will need… ❏ VM running Linux 2563sec host $ nmap -sT -p 1-1000 192.168.122.221 … Not shown: 999 closed ports PORT STATE SERVICE 22/tcp open ssh Nmap done: 1 IP address (1 host up) scanned in 0.07 seconds scan first 1000 ports forward Wikipedia - Packet flow in Netfilter and General Networking Receive path for local delivery Service dispatch with BPF socket lookup packet metadata BPF program lookup result 010 101 010 struct bpf_sk_lookup