How and When You Should Measure CPU Overhead of eBPF Programs Bryce Kahle, Datadog October 28, 2020 Why should I profile eBPF programs? CI variance tracking Tools kernel.bpf_stats_enabled kernel

in-kernel verifier ensures that eBPF programs are safe to run and a JIT compiler converts the bytecode to CPU architecture specific instructions for native execution efficiency. eBPF programs can be run at various Deriving rate limits based on number of available CPU cores or available memory can be misleading as well as the Cilium agent may be subject to CPU and memory constraints. For this reason, all API call network-latency Set CPU governor to performance The CPU scaling up and down can impact latency tests and lead to sub-optimal performance. To achieve maximum consistent performance. Set the CPU governor to performance:

clusters or clustermeshes with more than 65535 nodes. Decryption with Cilium IPsec is limited to a single CPU core per IPsec tunnel. This may affect performance in case of high throughput between two nodes. WireGuard in-kernel verifier ensures that eBPF programs are safe to run and a JIT compiler converts the bytecode to CPU architecture specific instructions for native execution efficiency. eBPF programs can be run at various Deriving rate limits based on number of available CPU cores or available memory can be misleading as well as the Cilium agent may be subject to CPU and memory constraints. For this reason, all API call

The data deluge on modern storage 2 Compute node CPU Network Storage node Flash The data deluge on modern storage 3 Compute node 3 CPU Network Storage node Flash 16-lane PCIe, 16GB/s eBPF and DoS 6 Compute node CPU Network Storage node Flash eBPF and DoS 7 Compute node CPU Network Storage node Flash DoS eBPF and DoS 8 Compute node CPU Network Storage node Flash DoS reverse! 9 Compute node CPU Network Storage node Flash DoS in reverse! 10 Compute node CPU Network Storage node Flash Data DoS in reverse! 11 Compute node CPU Network Storage node Flash

in-kernel verifier ensures that BPF programs are safe to run and a JIT compiler converts the bytecode to CPU architecture specific instructions for native execution efficiency. BPF programs can be run at various BPF datapath to perform more aggressive aggregation on packet forwarding related events to reduce CPU consumption while running cilium monitor. The automatic change only applies to the default ConfigMap Deriving rate limits based on number of available CPU cores or available memory can be misleading as well as the Cilium agent may be subject to CPU and memory constraints. For this reason, all API call

in-kernel verifier ensures that eBPF programs are safe to run and a JIT compiler converts the bytecode to CPU architecture specific instructions for native execution efficiency. eBPF programs can be run at various Deriving rate limits based on number of available CPU cores or available memory can be misleading as well as the Cilium agent may be subject to CPU and memory constraints. For this reason, all API call and kube-scheduler instances. The CPU, memory and disk size set for the workers might be different for your use case. You might have pods that require more memory or CPU available so you should design your

\ --min-cpu-platform "Intel Broadwell" \ kata-testing gcloud compute ssh kata-testing # While ssh'd into the VM: $ [ -z "$(lscpu|grep GenuineIntel)" ] && { echo "ERROR: Need an Intel CPU"; exit 1; kernel verifier ensures that BPF programs are safe to run and a JIT compiler converts the bytecode to CPU architecture specific instructions for native execution efficiency. BPF programs can be run at various BPF datapath to perform more aggressive aggregation on packet forwarding related events to reduce CPU consumption while running cilium monitor. The automatic change only applies to the default ConfigMap

in-kernel verifier ensures that BPF programs are safe to run and a JIT compiler converts the bytecode to CPU architecture specific instruc�ons for na�ve execu�on efficiency. BPF programs can be run at various between 10 seconds and 30 minutes or 12 hours for LRU based maps. This should automa�cally op�mize CPU consump�on as much as possible while keeping the connec�on tracking table u�liza�on below 25%. If needed and bpf-ct-global-tcp-max can be increased. Se�ng both of these op�ons will be a trade-off of CPU for conntrack-gc-interval , and for bpf-ct-global-any-max and bpf-ct-global-tcp-max the amount of

kernel verifier ensures that BPF programs are safe to run and a JIT compiler converts the bytecode to CPU architecture specific instructions for native execution efficiency. BPF programs can be run at various BPF datapath to perform more aggressive aggregation on packet forwarding related events to reduce CPU consumption while running cilium monitor. The automatic change only applies to the default ConfigMap between 10 seconds and 30 minutes or 12 hours for LRU based maps. This should automatically optimize CPU consumption as much as possible while keeping the connection tracking table utilization below 25%.

- Trace complex Ruby programs execution rbperf – on-CPU profiling - $ rbperf record --pid=124 cpu - $ rbperf report [...] rbperf – Rails on-CPU profile rbperf – tracing write(2) calls - $ rbperf