from the Data Deluge? A case for file filtering in eBPF Giulia Frascaria October 28, 2020 1 The data deluge on modern storage 2 Compute node CPU Network Storage node Flash The data deluge on CPU Network Storage node Flash Data DoS in reverse! 11 Compute node CPU Network Storage node Flash Data So similar yet so different ● DoS is malicious ● Data transfer is business-critical ● We 12 So similar yet so different ● DoS is malicious ● Data transfer is business-critical ● We can blindly drop DoS 13 But could we reduce data transfer size? eBPF filter-reduce 14 Filter Reduce input

structure and the supported formats. Monitoring & Metrics : Instructions for configuring metrics collection from Cilium. Troubleshooting : Describes how to troubleshoot Cilium in different deployment modes Cilium Hubble Important common packages Debugging toFQDNs and DNS Debugging Mutexes / Locks and Data Races Hubble Bumping the vendored Cilium dependency Documentation Style Header Titles Body Code Available Cilium Operator CRD Registration IPAM KVStore operations Identity garbage collection CiliumEndpoint garbage collection Derivative network policy creation Security Identities Reference Command Cheatsheet

structure and the supported formats. Monitoring & Metrics : Instructions for configuring metrics collection from Cilium. Troubleshooting : Describes how to troubleshoot Cilium in different deployment modes Cilium Hubble Important common packages Debugging toFQDNs and DNS Debugging Mutexes / Locks and Data Races Hubble Bumping the vendored Cilium dependency Release Management Organization Release tracking Available Cilium Operator CRD Registration IPAM KVStore operations Identity garbage collection CiliumEndpoint garbage collection Derivative network policy creation Reference Command Cheatsheet Command utilities:

structure and the supported formats. Monitoring & Metrics : Instructions for configuring metrics collection from Cilium. Troubleshooting : Describes how to troubleshoot Cilium in different deployment modes wildcards for the kubernetes block like this: kubectl -n kube-system edit cm coredns [...] apiVersion: v1 data: Corefile: | .:53 { errors health kubernetes cluster.local in-addr.arpa configuration: apiVersion: v1 kind: ConfigMap metadata: name: cni-configuration namespace: cilium data: cni-config: |- { "cniVersion": "0.3.0", "name": "azure", "plugins": [

structure and the supported formats. Monitoring & Metrics : Instructions for configuring metrics collection from Cilium. Troubleshooting : Describes how to troubleshoot Cilium in different deployment modes run in standalone mode or as a cluster making it a great choice for local testing with multi-node data paths. Agent nodes are joined to the master node using a node-token which can be found on the master wildcards for the kubernetes block like this: kubectl -n kube-system edit cm coredns [...] apiVersion: v1 data: Corefile: | .:53 { errors health kubernetes cluster.local in-addr.arpa

structure and the supported formats. Monitoring & Metrics : Instructions for configuring metrics collection from Cilium. Troubleshooting : Describes how to troubleshoot Cilium in different deployment modes Cilium Hubble Important common packages Debugging toFQDNs and DNS Debugging Mutexes / Locks and Data Races Hubble Bumping the vendored Cilium dependency Release Management Organization Release tracking Cilium’s eBPF implementation is optimized for maximum performance, can be attached to XDP (eXpress Data Path), and supports direct server return (DSR) as well as Maglev consistent hashing if the load balancing

structure and the supported formats. Monitoring & Metrics : Instructions for configuring metrics collection from Cilium. Troubleshooting : Describes how to troubleshoot Cilium in different deployment modes Cilium Hubble Important common packages Debugging toFQDNs and DNS Debugging Mutexes / Locks and Data Races Release Management Organization Release tracking Release Cadence Backporting process Backport run in standalone mode or as a cluster making it a great choice for local testing with multi-node data paths. Agent nodes are joined to the master node using a node-token which can be found on the master

bpf_stats_enabled kernel.bpf_stats_enabled sysctl – Added in kernel v5.1 (off by default) – Turns on stats collection for all eBPF programs – exposes total run_time_ns and run_cnt – Use cases: – Benchmarking + repeat – Control input data and/or context. Examine output data/context. – Use cases: – Unit testing – Debugging bpftool prog run Program Type Input Data Input Context Output Data Output Context Repeat

retrieve configuration options, and store state through eBPF maps to save and retrieve data in a wide set of data structures. These maps can be accessed from eBPF programs as well as from applications their own encoding. When the bees jumped on the case, it marked the beginning Of a whole new era for data sharing and messaging. Mail was still slow to go through the ship’s processors, But the electrician eBPF enables broad system observability. Collection and in-kernel aggregation of metrics allows flexible and efficient generation of observability events and data structures from a wide range of possible

2020 Source: https://pcp.io https://github.com/iovisor/bcc https://bpftrace.org 2 eBPF Compiler Collection bpftrace BCC high-level tracing language for eBPF Performance Co-Pilot system performance analysis applications and networks. 92 6000+ Agents Performance Metrics Domain Agents export performance data from the kernel, services (e.g. PostgreSQL) and other instrumented applications Metrics Metrics