eBPF for Cloud-Native Networking and Performance Engineering

Introduction: eBPF as the Networking Performance Frontier

Networking in cloud-native environments has hit an inflection point. The Kubernetes networking model -- with its reliance on iptables chains, userspace proxies, and overlay networks -- served the ecosystem well during its formative years. But as organizations push toward higher pod densities, lower tail latencies, and multi-terabit throughput requirements, the limitations of traditional Linux networking have become painfully visible. Connection tracking tables overflow. Iptables rule evaluation scales linearly with rule count. Sidecar proxies double the memory footprint of every workload. The networking stack that was "good enough" for a hundred pods collapses at ten thousand.

eBPF -- extended Berkeley Packet Filter -- has emerged as the answer to these scaling challenges. By allowing engineers to attach custom programs directly to kernel networking hooks, eBPF bypasses the traditional networking stack entirely when needed, processes packets at wire speed, and provides granular visibility into every network flow without the overhead of userspace context switches. What began as a packet filtering mechanism has become a full-fledged programmable data plane that powers the networking infrastructure at some of the largest companies on the planet.

This article is a deep technical exploration of eBPF for networking and performance engineering. We will walk through the core networking subsystems -- XDP, tc BPF, and socket-level programs -- before examining how these primitives are composed into production systems for load balancing, service mesh networking, network security, and performance analysis. We will cover cloud provider integrations, custom program development patterns, optimization techniques, and real-world production case studies. Whether you are a platform engineer evaluating eBPF-based CNIs, a network engineer looking to replace iptables at scale, or a performance engineer hunting microsecond-level latencies, this guide provides the technical depth you need.

The eBPF Networking Stack: XDP, tc BPF, and Socket Programs

eBPF provides multiple attachment points across the Linux networking stack, each operating at a different layer and offering different trade-offs between performance and flexibility. Understanding these hook points is fundamental to designing eBPF-based networking solutions.

XDP: Express Data Path

XDP (Express Data Path) is the earliest hook point in the Linux networking stack, executing eBPF programs before the kernel allocates an sk_buff structure for the incoming packet. This positioning is what gives XDP its extraordinary performance characteristics -- by processing packets before the kernel's networking stack touches them, XDP programs avoid the overhead of memory allocation, protocol parsing, and connection tracking that dominate traditional packet processing.

An XDP program receives a raw packet buffer and returns one of several verdict codes that control the packet's fate:

• XDP_DROP: Immediately discards the packet. This is the fastest possible path for filtering unwanted traffic and forms the basis of XDP-based DDoS mitigation.
• XDP_PASS: Passes the packet up to the normal kernel networking stack for standard processing.
• XDP_TX: Transmits the packet back out the same network interface it arrived on. This enables bounce-style architectures for load balancing and packet reflection.
• XDP_REDIRECT: Redirects the packet to a different network interface, a different CPU, or into an AF_XDP socket for userspace processing.
• XDP_ABORTED: Indicates an error condition and drops the packet while generating a tracepoint for debugging.

XDP programs operate in three modes that offer different performance and compatibility trade-offs:

Native XDP (driver mode) runs directly within the network driver's receive path, achieving the highest possible performance. The program executes before the kernel even allocates metadata structures for the packet, which means processing overhead is measured in nanoseconds rather than microseconds. Most modern NIC drivers (including mlx5, i40e, ice, bnxt, and virtio_net) support native XDP. On modern hardware with 100 Gbps NICs, native XDP can process upwards of 26 million packets per second per core.

Offloaded XDP takes this further by compiling eBPF programs directly onto the NIC's programmable hardware (SmartNICs like Netronome or NVIDIA BlueField). This offloads packet processing entirely from the host CPU, freeing compute resources for application workloads. While the eBPF instruction subset supported by offloaded mode is more limited, it enables true line-rate processing even on 100+ Gbps links.

Generic XDP (SKB mode) runs later in the stack, after the sk_buff has been allocated. It provides XDP functionality on any network driver but sacrifices the performance benefits of early processing. Generic XDP is primarily useful for development and testing on hardware that lacks native XDP support.

tc BPF: Traffic Control Programs

While XDP operates at the ingress path before sk_buff allocation, tc (traffic control) BPF programs attach to the kernel's traffic control layer, which sits after sk_buff creation. tc BPF programs can be attached at both ingress and egress points, making them essential for scenarios where XDP alone is insufficient -- particularly for outbound traffic shaping, packet modification that requires full sk_buff context, and forwarding decisions that depend on higher-layer protocol information.

tc BPF programs have access to the full sk_buff structure, which provides richer metadata than XDP's raw packet buffer. This includes information about the socket, the routing decision, connection tracking state, and netfilter marks. The trade-off is higher latency per packet compared to XDP, but tc BPF is still dramatically faster than equivalent iptables rules.

In Cilium's architecture, tc BPF programs handle the majority of the networking logic. When a packet enters a pod's virtual ethernet interface, a tc BPF program on the host side performs identity lookup, applies network policy, handles NAT, and makes forwarding decisions -- all within a single program execution. This replaces what would traditionally require traversal through multiple iptables chains, conntrack lookups, and netfilter hooks.

Socket-Level BPF Programs

Beyond packet-level processing, eBPF provides attachment points at the socket layer that enable manipulation of network connections at a higher level of abstraction:

cgroup/connect4 and cgroup/connect6 programs intercept connect() system calls, allowing transparent redirection of connections. Cilium uses these programs to implement Kubernetes service load balancing at the socket level -- when a pod calls connect() to a ClusterIP service address, the eBPF program transparently rewrites the destination to a healthy backend pod, avoiding the overhead of NAT and conntrack entirely.

sk_msg and sk_skb programs enable socket-level packet redirection. When two pods on the same node communicate through a Kubernetes service, the traditional path involves the packet traversing the full networking stack twice (out of the source pod and into the destination pod). With sk_msg programs attached via a sockmap, packets can be redirected directly from one socket's send buffer to another socket's receive buffer, bypassing the entire networking stack. This shortcutting can reduce latency by 50 percent or more for node-local traffic.

SO_ATTACH_BPF and SO_ATTACH_REUSEPORT_EBPF programs enable custom socket filtering and load balancing at the socket layer, allowing fine-grained control over which packets reach userspace.

eBPF-Based Load Balancing: Replacing kube-proxy and Beyond

One of the highest-impact applications of eBPF networking is replacing Kubernetes' default service load balancing, which relies on kube-proxy and iptables (or IPVS). This replacement delivers dramatic performance improvements at scale and has become a primary driver of eBPF adoption in production Kubernetes clusters.

The kube-proxy Problem

Kubernetes services provide a stable virtual IP (ClusterIP) that load balances traffic across a set of backend pods. The default implementation, kube-proxy, programs these load balancing rules into the host's iptables or IPVS tables. While this approach works correctly, it has well-documented scaling problems.

In iptables mode, kube-proxy creates a chain of rules for each service and backend. When a packet arrives destined for a service ClusterIP, the kernel evaluates these chains sequentially using probability-based rules to distribute traffic. With 5,000 services, each with 10 endpoints, this creates approximately 50,000 iptables rules. Every packet destined for any service must traverse a significant portion of these rules. Rule updates require regenerating and atomically swapping the entire iptables ruleset, which in large clusters can take 5 to 10 seconds and cause noticeable CPU spikes.

IPVS mode improves on this by using hash-based lookups instead of linear chain traversal, but it still operates within the netfilter framework, requires conntrack for NAT, and adds its own complexity around service health checking and session affinity.

Cilium's kube-proxy Replacement

Cilium's eBPF-based kube-proxy replacement eliminates these scaling limitations by implementing service load balancing entirely in eBPF maps and programs. The architecture works as follows:

The Cilium agent watches the Kubernetes API server for Service and Endpoint resources and populates eBPF hash maps with the mapping from service virtual IPs and ports to backend pod IPs and ports. When a packet destined for a service VIP enters the data path, a tc BPF program performs an O(1) hash lookup to find the service, selects a backend using a configurable algorithm (random, Maglev consistent hashing, or weighted round-robin), and rewrites the packet headers inline. No iptables rules, no conntrack entries for the DNAT, no netfilter overhead.

For node-local traffic, Cilium goes further. Using cgroup/connect4 eBPF programs, it intercepts the connect() system call and performs service resolution at the socket level. The application's connection is transparently directed to the backend pod's actual IP address, meaning the packet never carries the service VIP at all. This eliminates the need for DNAT and reverse SNAT, avoids creating conntrack entries entirely, and reduces latency for every connection to a Kubernetes service.

The Maglev consistent hashing implementation is particularly important for production deployments. Based on the algorithm described in Google's Maglev paper, it provides consistent backend selection across nodes without requiring shared state. When a backend pod is added or removed, only a minimal fraction of connections are remapped, making it ideal for stateful protocols and long-lived connections.

Katran: Meta's L4 Load Balancer

Meta (formerly Facebook) developed Katran, an open-source L4 load balancer built entirely on XDP. Katran sits at the edge of Meta's network, distributing incoming traffic across pools of L7 load balancers. Its architecture demonstrates the extreme performance characteristics achievable with eBPF.

Katran uses XDP programs to perform encapsulation-based load balancing. Incoming packets are parsed, matched against a virtual IP configuration, assigned to a backend using consistent hashing, and encapsulated in a GUE (Generic UDP Encapsulation) or IPIP tunnel to the selected backend -- all within a single XDP program execution. The backend decapsulates the outer header and processes the original packet normally.

This design allows a single Katran instance to handle millions of connections per second. Because XDP processes packets before the kernel networking stack, Katran avoids all of the overhead associated with socket buffers, connection tracking, and routing lookups. Meta reports that Katran processes packets in under 100 nanoseconds per packet on average, enabling a single server to handle the full traffic load of a datacenter edge without breaking a sweat.

Katran's source code is available on GitHub, and its architecture has influenced the design of eBPF-based load balancers across the industry. Several cloud providers have adopted similar XDP-based approaches for their managed load balancing services.

Comparing Load Balancing Approaches

The performance differences between iptables, IPVS, and eBPF-based load balancing become stark at scale:

These numbers represent per-packet overhead in a cluster with 10,000 services. The socket-level eBPF approach achieves the lowest overhead because it avoids packet manipulation entirely -- the connection is directed to the correct backend from the moment the socket is created.

Network Traffic Analysis and Flow Visibility

Production network debugging in Kubernetes has historically been a nightmare. Packets traverse multiple network namespaces, virtual ethernet pairs, bridges, overlay encapsulations, and NAT translations. Traditional tools like tcpdump can capture packets at a single point, but correlating flows across the entire path from source pod to destination pod requires heroic effort. eBPF changes this fundamentally by providing kernel-level visibility into every network decision.

Hubble: Network Observability for Cilium

Hubble is Cilium's observability layer, built on eBPF to provide comprehensive network flow visibility. Unlike traditional flow exporters that operate at the interface level, Hubble captures flows at the eBPF data path level, where it has access to Cilium's identity-aware metadata. This means every flow is annotated not just with IP addresses and ports, but with Kubernetes pod names, namespace labels, service names, and security identity information.

Hubble operates through a per-node agent that receives flow events from Cilium's eBPF data path via a perf event ring buffer. These events include connection establishment, packet drops (with drop reasons), policy verdicts, DNS query/response pairs, and L7 protocol events for HTTP, gRPC, and Kafka when L7 visibility is enabled.

The flow data Hubble collects is extraordinarily detailed. For every network connection, you can see:

• Source and destination identity (pod, namespace, labels, security identity)
• Whether the connection was forwarded, dropped, or redirected
• The specific network policy that allowed or denied the flow
• DNS queries and their responses, including TTL and resolution latency
• TCP connection state transitions and retransmission events
• L7 protocol metadata (HTTP method, path, status code; gRPC service and method; Kafka topic and partition)

Hubble Relay aggregates flow data across all nodes in the cluster, providing cluster-wide visibility through a single API endpoint. The Hubble UI provides a real-time service dependency map that is generated entirely from observed network flows -- no manual configuration or service registration required.

DNS Monitoring with eBPF

DNS is the backbone of service discovery in Kubernetes, and DNS failures are one of the most common causes of application errors in production. eBPF-based DNS monitoring provides visibility that traditional approaches cannot match.

Cilium's eBPF data path intercepts DNS queries and responses at the packet level, parsing the DNS protocol inline within the eBPF program. This provides complete visibility into DNS resolution behavior: which pods are querying which domains, how long resolution takes, which queries result in NXDOMAIN or SERVFAIL responses, and whether DNS responses contain unexpected entries that might indicate DNS poisoning.

Beyond passive monitoring, eBPF-based DNS interception enables active DNS-aware network policies. Cilium can enforce policies based on DNS-resolved domain names rather than IP addresses, which is critical for controlling egress traffic to external services. When a pod resolves api.stripe.com, Cilium's eBPF programs capture the returned IP addresses and dynamically populate allow-lists for those IPs, ensuring that the pod can only reach the actual IPs returned by DNS for the allowed domain.

Flow Logs at Scale

One of the persistent challenges with network flow logging is the sheer volume of data generated in busy clusters. A cluster handling 100,000 requests per second generates millions of flow events per minute, which can overwhelm logging infrastructure and storage.

eBPF-based flow collection addresses this through in-kernel aggregation. Rather than exporting a flow event for every packet, eBPF programs maintain per-flow state in BPF hash maps, aggregating packet counts, byte counts, and timing information. Flow records are exported to userspace only at configurable intervals or on flow completion, reducing the data volume by orders of magnitude while preserving the essential information needed for debugging and compliance.

Cilium's Hubble implementation supports configurable flow sampling, per-identity filtering, and DNS-aware flow aggregation, allowing operators to tune the trade-off between visibility and overhead for their specific requirements.

eBPF for Service Mesh Data Planes

The service mesh landscape has been reshaped by eBPF. The traditional sidecar proxy model -- where every pod runs an Envoy proxy that intercepts all network traffic -- served the industry well as an initial architecture, but its costs have become increasingly difficult to justify. Every sidecar proxy consumes 50 to 100 megabytes of memory, adds 1 to 3 milliseconds of latency per hop, and doubles the number of TCP connections in the cluster. For organizations running thousands of pods, the overhead is substantial.

Cilium Service Mesh: The Sidecar-less Architecture

Cilium Service Mesh implements a fundamentally different architecture. Instead of deploying a sidecar proxy in every pod, Cilium handles L3/L4 service mesh functionality (mutual TLS, load balancing, circuit breaking, retry logic, connection pooling) entirely within the eBPF data path at the kernel level. Only L7 traffic management features that require full protocol parsing (HTTP header-based routing, gRPC load balancing, rate limiting based on request content) are delegated to a shared Envoy proxy instance running per-node rather than per-pod.

This architecture dramatically reduces resource consumption. Instead of N sidecar proxies (one per pod), you have one shared Envoy proxy per node. L4 traffic -- which constitutes the majority of network flows in most clusters -- never leaves the kernel, avoiding the latency and CPU overhead of userspace proxy processing entirely.

The eBPF data path in Cilium Service Mesh handles several functions that traditionally required a sidecar proxy:

Transparent encryption with WireGuard or IPsec: Cilium can encrypt all pod-to-pod traffic at the kernel level using WireGuard, providing mutual authentication and confidentiality without the TLS overhead associated with sidecar-based mTLS. WireGuard operates at L3, encrypting entire IP packets with minimal CPU overhead -- typically under 5 percent even at high throughput.

L4 load balancing and connection management: Service-to-service connections are load balanced using eBPF maps, with support for consistent hashing, weighted backends, and active health checking. Connection state is tracked in eBPF conntrack maps, and failed connections are retried to alternate backends transparently.

Network policy enforcement: eBPF programs enforce both Kubernetes NetworkPolicy and Cilium's extended CiliumNetworkPolicy at the kernel level, with identity-based allow/deny decisions made in nanoseconds rather than the milliseconds required for iptables chain evaluation.

Bandwidth management: eBPF-based rate limiting using EDT (Earliest Departure Time) scheduling provides precise bandwidth control for pods, replacing the blunt tc-tbf and tc-htb qdiscs with a mechanism that integrates directly with the kernel's packet scheduler for smoother traffic shaping and better burst handling.

Kernel-Level L7 Processing

An exciting development in the eBPF service mesh space is the push toward handling more L7 protocol processing directly in eBPF, reducing the need for even the per-node Envoy proxy. Recent kernel versions (6.x and beyond) have expanded the capabilities available to eBPF programs, including larger program sizes, more complex data structure support, and improved string processing -- all of which are prerequisites for parsing HTTP headers and routing requests within eBPF.

Several experimental projects have demonstrated HTTP/1.1 request routing implemented entirely in eBPF tc programs. While these are not yet production-ready for general use, they point toward a future where the entire service mesh data plane, including L7 routing, operates at kernel speed. The primary challenges remain program complexity limits, the difficulty of handling connection-oriented protocols (particularly HTTP/2 and gRPC with their multiplexed streams), and the need for TLS termination which currently requires userspace processing.

Performance Impact of Sidecar Elimination

The performance benefits of the sidecar-less architecture are measurable and significant in production:

The latency reduction is particularly important for microservice architectures where a single user request might traverse 10 to 20 services. A 2 millisecond overhead per hop accumulates to 40 milliseconds across a 20-hop request path -- with eBPF-based processing, that overhead drops to under 6 milliseconds.

Performance Engineering with eBPF

While eBPF's networking capabilities get much of the attention, its impact on performance engineering is equally transformative. eBPF provides the ability to instrument virtually any kernel or userspace function without restarting processes, recompiling code, or deploying new agents. This makes it an invaluable tool for diagnosing the subtle, intermittent performance issues that plague production systems.

CPU Profiling and Flame Graphs

Traditional CPU profiling relies on periodic sampling of the call stack, typically using perf to capture stack traces at fixed intervals. eBPF enhances this approach by allowing profiles to be captured with kernel-level precision and filtered in-kernel before any data reaches userspace.

eBPF-based CPU profilers attach to the perf_event subsystem's timer interrupt, capturing stack traces at configurable frequencies (typically 49 or 99 Hz to avoid synchronization artifacts). The captured stack traces are stored in eBPF stack trace maps and aggregated in-kernel using BPF hash maps, with stack trace hashes as keys and occurrence counts as values. Only the aggregated data is transferred to userspace for flame graph generation, reducing the data volume by orders of magnitude compared to raw perf record output.

This in-kernel aggregation is what makes continuous production profiling practical. Tools like Parca and Grafana Pyroscope leverage eBPF to run always-on CPU profilers with under 1 percent overhead, capturing the data needed to generate flame graphs that pinpoint exactly where CPU time is being spent across every process on the system.

Off-CPU Analysis

CPU profiling tells you where time is spent executing code, but in modern systems, the majority of latency often comes from time spent not running on the CPU -- waiting for I/O, blocked on locks, sleeping on futexes, or waiting for network responses. eBPF excels at off-CPU analysis because it can instrument the exact kernel functions where threads transition between running and sleeping states.

An off-CPU analysis eBPF program typically attaches kprobes to finish_task_switch (or the equivalent scheduler tracepoint sched:sched_switch), capturing the stack trace and timestamp when a thread is descheduled. When the thread is scheduled back onto a CPU, the program calculates the duration of the off-CPU period and records it alongside the stack trace that led to the block. This produces off-CPU flame graphs that reveal the hidden sources of latency in your application.

Common findings from off-CPU analysis include:

• Lock contention: Threads blocked in pthread_mutex_lock or kernel mutex_lock, revealing contention on shared data structures
• I/O waits: Threads blocked in io_schedule or wait_for_completion, showing which I/O operations are slow
• Network waits: Threads blocked in sk_wait_data or inet_csk_wait_for_connect, revealing slow network dependencies
• Futex contention: Threads blocked in futex_wait, common in Go and Java runtimes where runtime-level synchronization maps to futex operations
• Page faults: Threads blocked in handle_mm_fault during major page faults, indicating memory pressure or inefficient memory mapping

Scheduler Tracing and Runqueue Latency

In high-performance systems, even small scheduling delays can impact tail latency. eBPF enables precise measurement of runqueue latency -- the time a thread spends waiting to be scheduled onto a CPU after it becomes runnable.

By attaching to sched:sched_wakeup and sched:sched_switch tracepoints, an eBPF program can measure the exact duration between when a thread is woken up (placed on the runqueue) and when it actually starts executing. In well-tuned systems, this should be under 10 microseconds. When runqueue latency climbs above 100 microseconds, it indicates CPU contention -- too many runnable threads competing for available cores.

Brendan Gregg's runqlat tool from the BCC toolkit uses this technique to produce histograms of runqueue latency across the system, making it straightforward to identify scheduling bottlenecks. The eBPF program runs entirely in kernel space, with the histogram aggregation happening in-kernel via BPF maps, so the measurement overhead is negligible even on heavily loaded systems.

Lock Contention Analysis

Lock contention is one of the most difficult performance problems to diagnose with traditional tools. eBPF provides several approaches:

Kernel lock analysis uses tracepoints in the kernel's locking subsystem (lock:contention_begin, lock:contention_end) to measure contention on kernel mutexes, spinlocks, and rwlocks. This reveals internal kernel bottlenecks that affect application performance -- for example, contention on the mmap_lock during memory allocation, or contention on inode locks during file operations.

Userspace lock analysis uses uprobes to instrument pthread mutex functions, measuring the time spent blocked on each lock. By capturing the stack trace at lock acquisition, you can identify exactly which code paths are contending and which locks are the bottleneck.

The combination of CPU profiling, off-CPU analysis, and lock contention analysis provides a comprehensive view of where time is spent in production systems. This holistic approach is only practical with eBPF because the instrumentation overhead is low enough to run continuously in production rather than requiring reproduction in test environments.

Network Security with eBPF

eBPF has become a foundational technology for network security in Kubernetes, providing enforcement mechanisms that are both more performant and more expressive than traditional iptables-based approaches.

Identity-Based Network Policies

The fundamental limitation of IP-based network policies is that IP addresses in Kubernetes are ephemeral. Pods are created and destroyed constantly, IP addresses are reassigned, and by the time a security incident is investigated, the IP address involved may belong to a completely different workload. eBPF enables identity-based networking, where security decisions are made based on cryptographic identities rather than IP addresses.

In Cilium's implementation, each workload is assigned a security identity derived from its Kubernetes labels. This identity is encoded in network packets (either in the packet header for same-cluster traffic or via SPIFFE/mTLS for cross-cluster traffic) and evaluated by eBPF programs at the kernel level. Network policies are expressed in terms of these identities:

apiVersion: cilium.io/v2
kind: CiliumNetworkPolicy
metadata:
  name: api-isolation
spec:
  endpointSelector:
    matchLabels:
      app: payment-api
  ingress:
    - fromEndpoints:
        - matchLabels:
            app: checkout-service
      toPorts:
        - ports:
            - port: '8080'
              protocol: TCP
          rules:
            http:
              - method: POST
                path: '/v1/charges'

This policy allows only the checkout-service to reach the payment-api, and only via HTTP POST requests to the /v1/charges endpoint. The L3/L4 portion of this policy is enforced entirely within eBPF at the kernel level, while the L7 HTTP matching is delegated to the per-node Envoy proxy.

Microsegmentation at Scale

Traditional microsegmentation -- restricting network communication between individual workloads -- is prohibitively expensive with iptables because each policy requires multiple rules, and the total rule count grows combinatorially with the number of workloads and policies. In a cluster with 10,000 pods and fine-grained microsegmentation policies, the iptables rule count can reach hundreds of thousands, causing measurable CPU overhead on every packet.

eBPF-based microsegmentation avoids this scaling problem entirely. Policy decisions are encoded in BPF hash maps keyed by security identity pairs (source identity, destination identity, destination port), making each policy lookup O(1) regardless of the total number of policies. Adding a new policy requires inserting a single entry into the map, rather than rebuilding the entire iptables ruleset.

Cilium supports both Kubernetes-native NetworkPolicy and its own CiliumNetworkPolicy and CiliumClusterwideNetworkPolicy resources. The Cilium policy engine supports identity-based selectors, CIDR-based selectors, DNS-based selectors (for egress to external services), and L7 protocol-aware rules for HTTP, gRPC, Kafka, and DNS.

Connection Tracking and NAT

eBPF-based connection tracking replaces the kernel's nf_conntrack subsystem for connections managed by Cilium. The BPF conntrack implementation uses per-CPU hash maps to track connection state, avoiding the lock contention that plagues nf_conntrack in high-connection-rate environments.

Traditional nf_conntrack uses a single hash table protected by per-bucket spinlocks, which can become a severe bottleneck on systems handling hundreds of thousands of new connections per second. Each new connection requires locking a hash bucket, inserting an entry, and (for NAT) locking and inserting a second entry for the reverse direction. Under heavy load, this lock contention becomes a primary source of packet drops and latency spikes.

Cilium's BPF conntrack uses per-CPU maps, eliminating lock contention entirely since each CPU core operates on its own independent map. NAT state is stored alongside connection tracking state in the same map entry, avoiding the separate NAT table lookups required by nf_conntrack. The result is connection establishment that scales linearly with CPU cores, handling millions of new connections per second on modern hardware without the conntrack-related packet drops that plague high-traffic iptables-based setups.

eBPF in Cloud Provider Networking

The major cloud providers have embraced eBPF as a core component of their managed Kubernetes networking offerings. This integration brings eBPF's performance and visibility benefits to managed clusters without requiring users to deploy and manage Cilium independently.

AWS: VPC CNI with eBPF

Amazon EKS supports eBPF-based networking through two paths. The AWS VPC CNI plugin has added eBPF-based network policy enforcement, allowing EKS users to enforce Kubernetes NetworkPolicy using eBPF programs rather than iptables. This integration leverages the VPC CNI's direct VPC networking model (where pods receive real VPC IP addresses) while adding eBPF-based policy enforcement for traffic between pods.

For organizations wanting the full eBPF networking stack, EKS supports Cilium as a CNI plugin, with documented deployment guides and tested compatibility with EKS-specific features like pod security groups, VPC flow logs, and AWS Load Balancer Controller integration.

AWS has also integrated eBPF into its Nitro network architecture, using eBPF programs within the Nitro hypervisor for network function virtualization, traffic shaping, and security group enforcement. While the specifics are not publicly documented, AWS has presented at multiple conferences about their use of eBPF/XDP for high-performance packet processing in the Nitro platform.

Azure: CNI Powered by Cilium

Azure Kubernetes Service (AKS) has made the most explicit commitment to eBPF-based networking among the major cloud providers. "Azure CNI Powered by Cilium" is the recommended networking option for new AKS clusters, providing eBPF-based pod networking, network policy enforcement, and kube-proxy replacement out of the box.

The Azure integration is particularly deep. Cilium runs as the primary CNI plugin, handling pod IP address management via Azure IPAM, and the eBPF data path replaces both kube-proxy and the traditional Azure network policy enforcement mechanism. Azure has contributed engineering resources to the Cilium project and maintains a fork with Azure-specific enhancements for VNET integration, accelerated networking (SR-IOV) compatibility, and Azure Network Policy Manager interop.

AKS clusters using Azure CNI Powered by Cilium benefit from Hubble network observability integrated into Azure Monitor, providing network flow visibility directly in the Azure portal without requiring separate observability infrastructure.

Google Cloud: GKE Dataplane V2

Google Kubernetes Engine's Dataplane V2 is built on Cilium and has been the default data plane for new GKE clusters since 2023. Dataplane V2 uses eBPF for all pod networking, service load balancing (replacing kube-proxy), and network policy enforcement.

Google's integration adds GKE-specific enhancements, including integration with VPC-native pod addressing, Google Cloud Armor for DDoS protection, and Cloud NAT for egress traffic. GKE Dataplane V2 also provides built-in network policy logging, capturing allow and deny decisions for compliance and debugging.

The adoption of eBPF across all three major cloud providers signals the technology's maturity. For organizations running managed Kubernetes, eBPF networking is no longer an experimental choice -- it is the default path recommended by cloud providers themselves.

Custom eBPF Program Development for Networking

While tools like Cilium provide comprehensive eBPF networking solutions out of the box, many organizations need custom eBPF programs for specialized networking requirements. The development ecosystem has matured significantly, with multiple frameworks providing high-level abstractions over the raw eBPF APIs.

libbpf and the CO-RE Revolution

libbpf is the canonical C library for eBPF program development, maintained as part of the Linux kernel source tree. The most important evolution in libbpf has been CO-RE (Compile Once, Run Everywhere), which solves the portability challenge that previously plagued eBPF development.

Before CO-RE, eBPF programs that accessed kernel data structures needed to be compiled on the exact kernel version where they would run, because field offsets within structures change between kernel versions. CO-RE uses BTF (BPF Type Format) -- debug information embedded in modern kernels -- to relocate field accesses at load time, allowing a single compiled eBPF program to run on any kernel that supports BTF. This is similar in concept to how Java bytecode runs on any JVM, but at the kernel level.

A typical CO-RE XDP program for packet filtering follows this pattern:

1. Define the eBPF program in C, using bpf_core_read() macros for portable kernel structure access
2. Compile to eBPF bytecode using clang with the -target bpf flag
3. Generate a skeleton header using bpftool gen skeleton, which provides a typed C API for loading and interacting with the program
4. Write a userspace loader that opens the skeleton, loads the programs into the kernel, and attaches them to the appropriate hooks

Writing XDP Programs

An XDP program for basic packet filtering demonstrates the core concepts. The program parses Ethernet and IP headers, checks the source IP against a BPF hash map of blocked addresses, and returns XDP_DROP for blocked traffic or XDP_PASS for allowed traffic.

Key considerations for production XDP programs include:

Bounds checking: The eBPF verifier requires that every memory access is explicitly bounds-checked against the packet data and data_end pointers. Failing to check bounds will cause the verifier to reject the program. This is the most common source of frustration for new eBPF developers, but it is also what makes eBPF programs provably safe to run in the kernel.

Header parsing: XDP programs receive a raw packet buffer without any protocol parsing. The program must walk the headers manually -- Ethernet header first, then IP (checking the IP version and header length), then TCP/UDP. Each step requires bounds checking. Helper functions and libraries like xdp-tools provide reusable parsing routines that handle common protocols.

Map interactions: XDP programs communicate with userspace through BPF maps. The most common map types for networking are hash maps (for lookup tables like IP blocklists), LPM trie maps (for CIDR-based routing), and per-CPU arrays (for statistics counters that avoid lock contention).

Writing tc BPF Programs

tc BPF programs follow a similar structure to XDP programs but operate on __sk_buff rather than xdp_md, providing richer context. tc programs are essential for:

• Egress processing: XDP only operates on ingress; tc BPF handles outbound traffic
• Packet modification requiring sk_buff: Some modifications (like adjusting checksums or modifying encapsulation) are easier with the sk_buff helpers available to tc programs
• Integration with the routing stack: tc programs can read and modify routing decisions, set packet marks, and interact with the connection tracking subsystem

tc programs are attached using the bpf_tc_attach API or the tc command-line tool. In modern kernels, the preferred attachment method uses TCX (TC eXpress), which provides a more efficient attachment mechanism than the legacy tc/cls_bpf approach.

The Aya Framework for Rust

For teams that prefer Rust over C, the Aya framework provides a pure-Rust eBPF development experience. Aya compiles Rust code to eBPF bytecode using LLVM's BPF backend, provides Rust-idiomatic APIs for maps and program types, and includes a userspace library for loading and managing eBPF programs.

Aya's advantages include Rust's memory safety guarantees (catching certain classes of bugs at compile time that would only be caught by the eBPF verifier at load time), the Rust ecosystem's strong tooling (cargo, clippy, rust-analyzer), and the ability to share types between the eBPF program and the userspace loader using a common Rust crate.

The framework has reached production maturity, with organizations like Datadog and Deepfence using Aya-based eBPF programs in their commercial products. The aya-rs project provides complete examples for XDP, tc, socket, and tracing programs that serve as excellent starting points.

eBPF Performance Optimization Techniques

Writing correct eBPF programs is one challenge; writing performant eBPF programs is another. The constrained environment of the eBPF virtual machine requires different optimization strategies than traditional userspace programming.

Tail Calls and BPF-to-BPF Function Calls

Complex networking programs often exceed what can be comfortably implemented in a single eBPF program. Two mechanisms address this:

Tail calls allow one eBPF program to jump to another, replacing the current program's execution context. The called program starts fresh with the same packet context but independent stack space. Tail calls are implemented via bpf_tail_call() using a program array map, and the kernel supports a maximum chain depth of 33 tail calls. Cilium uses tail calls extensively to decompose its networking pipeline into modular stages -- policy evaluation, NAT, encapsulation, and forwarding each run as separate tail-called programs.

BPF-to-BPF function calls (also called static functions) allow code reuse within a single eBPF program. Unlike tail calls, function calls share the calling program's stack and can return values. The verifier inlines small functions and uses actual call instructions for larger ones. Function calls are ideal for shared parsing routines, checksum calculations, and other utility code.

The choice between tail calls and function calls depends on the use case. Tail calls provide modularity and separate verification (each program is verified independently), making them ideal for composing pipeline stages. Function calls provide tighter integration and lower overhead for shared utility code within a single program.

Per-CPU Maps and Lock-Free Data Structures

In multi-core networking, data structure access patterns dramatically impact performance. Standard BPF hash maps use per-bucket spinlocks for concurrent access, which can become bottlenecks when multiple cores update the same map simultaneously.

Per-CPU maps (BPF_MAP_TYPE_PERCPU_HASH, BPF_MAP_TYPE_PERCPU_ARRAY) maintain separate copies of each entry per CPU core, eliminating all lock contention. They are ideal for statistics counters, per-flow state in environments where flow affinity ensures each flow is processed by a single core, and any data structure where exact cross-CPU consistency is not required.

For statistics, the userspace reader sums the per-CPU values to get the aggregate. This trades memory (N copies of each entry, where N is the CPU count) for performance (zero synchronization overhead).

Ring Buffers vs. Perf Buffers

eBPF programs communicate events to userspace through either perf buffers or the newer BPF ring buffer:

Perf buffers (BPF_MAP_TYPE_PERF_EVENT_ARRAY) provide per-CPU ring buffers that avoid cross-CPU synchronization. However, they waste memory (each CPU's buffer must be sized for peak load, even if most CPUs are idle) and make event ordering across CPUs non-deterministic.

BPF ring buffers (BPF_MAP_TYPE_RINGBUF), introduced in Linux 5.8, provide a single shared ring buffer across all CPUs. Despite the shared access, the ring buffer uses a lock-free design that achieves performance comparable to per-CPU perf buffers while using significantly less memory and preserving event ordering. For most new eBPF programs, the ring buffer is the preferred choice.

The ring buffer also supports two operation modes: bpf_ringbuf_output() copies data into the ring buffer (simpler but requires a copy), while bpf_ringbuf_reserve() / bpf_ringbuf_submit() allows the eBPF program to write directly into the ring buffer memory (zero-copy, but requires careful handling of the reserved slot).

Map-in-Map and Atomic Updates

For networking configurations that must be updated atomically (like load balancer backend lists or routing tables), map-in-map provides a powerful pattern. A map-in-map is a BPF map whose values are file descriptors pointing to other BPF maps. The outer map acts as an indirection layer -- to atomically update the configuration, you create a new inner map with the updated configuration and swap the outer map's pointer using bpf_map_update_elem(). In-flight packets continue using the old inner map until they complete, while new packets see the new configuration immediately.

Cilium uses this pattern for its service backend tables, ensuring that load balancer configuration updates are atomic and never cause a packet to see a partially-updated state.

Benchmarking eBPF Networking

Quantifying the performance benefits of eBPF networking requires rigorous benchmarking methodology. The differences between eBPF and traditional approaches are most pronounced under high load, where the per-packet overhead compounds into measurable throughput and latency impacts.

Throughput Benchmarks

In synthetic throughput tests using tools like iperf3 and netperf, eBPF-based networking consistently outperforms iptables-based approaches:

Single-stream TCP throughput shows modest differences because the bottleneck is typically the TCP stack itself rather than the packet processing path. Both iptables and eBPF approaches saturate 100 Gbps links in single-stream tests.

Multi-stream throughput with many services is where the difference becomes dramatic. With 10,000 Kubernetes services configured, iptables-based kube-proxy shows throughput degradation of 15 to 20 percent compared to a baseline with no services, due to the cumulative overhead of rule evaluation. Cilium's eBPF implementation shows under 1 percent degradation regardless of service count, because the O(1) hash lookup cost is constant.

Packets per second (small packet throughput) is the most sensitive benchmark. With 64-byte UDP packets, the per-packet overhead dominates. iptables-based processing achieves approximately 2 to 3 million packets per second per core. tc BPF achieves 8 to 12 million. XDP achieves 20 to 26 million. The difference reflects the amount of kernel code executed per packet -- XDP's early attachment point means most of the kernel networking stack is bypassed entirely.

Latency Benchmarks

Latency measurements, particularly tail latency (p99 and p99.9), reveal the impact of eBPF on request-level performance:

Pod-to-service latency with iptables-based kube-proxy adds 3 to 5 microseconds per hop for the DNAT and conntrack operations. With Cilium's socket-level eBPF load balancing, the overhead drops to under 500 nanoseconds because no NAT or conntrack is needed -- the connection is directed to the backend at the socket level.

Pod-to-pod latency on the same node benefits dramatically from sockmap-based shortcutting. The traditional path through the veth pair, bridge, and back through another veth pair adds approximately 10 microseconds. With eBPF sockmap redirection, packets bypass the entire networking stack, achieving latencies of 2 to 3 microseconds.

Tail latency under load is where eBPF's advantages compound. Under 80 percent CPU utilization, iptables-based networking shows p99.9 latency spikes of 500 microseconds or more due to lock contention in nf_conntrack and iptables rule evaluation. eBPF-based networking maintains p99.9 latency under 50 microseconds because the per-CPU data structures eliminate lock contention entirely.

CPU Overhead

CPU overhead measurements compare the percentage of CPU time spent on networking between approaches:

At a constant 1 million packets per second, iptables-based processing consumes approximately 12 percent of a single core. tc BPF consumes approximately 3 percent. XDP consumes under 1 percent. For organizations running thousands of pods with high inter-service communication rates, this CPU savings translates directly into either cost reduction (smaller node instances) or increased headroom for application workloads.

The chart above shows a typical CPU utilization breakdown on a Kubernetes node using eBPF-based networking, demonstrating that the networking overhead is a small fraction of total CPU usage.

Production Case Studies

The theoretical benefits of eBPF networking are compelling, but the real validation comes from production deployments at scale. Three organizations -- Meta, Cloudflare, and Netflix -- demonstrate different aspects of eBPF networking in demanding production environments.

Meta: Networking at Hyperscale

Meta operates one of the largest eBPF deployments in the world, using the technology across their entire networking stack from edge load balancing to inter-datacenter traffic management.

Katran handles all external-facing L4 load balancing at Meta's edge, processing billions of packets per day using XDP. Each Katran instance handles the traffic that would traditionally require dedicated hardware load balancers, at a fraction of the cost and with greater flexibility. Meta has reported that replacing hardware load balancers with Katran reduced their edge infrastructure costs significantly while improving the speed of configuration changes from hours to seconds.

bpfilter replaces iptables for host-level firewall rules across Meta's fleet. By translating iptables rules into eBPF programs, Meta gained the familiar iptables management interface while eliminating the performance overhead of the iptables evaluation engine. This was critical for their infrastructure because some hosts had thousands of firewall rules that were causing measurable packet processing delays.

Meta also uses eBPF extensively for network diagnostics and debugging. Custom eBPF programs trace packet paths through the kernel networking stack, identify conntrack table overflow events, measure per-flow retransmission rates, and detect asymmetric routing problems. These diagnostic capabilities are deployed fleet-wide and activate on-demand, meaning engineers can investigate networking problems on any machine in any datacenter without deploying additional software.

Cloudflare: DDoS Mitigation at the Edge

Cloudflare processes an enormous volume of network traffic across their global edge network, and eBPF is central to their DDoS mitigation strategy.

Cloudflare's DDoS mitigation pipeline uses XDP programs to inspect and filter traffic at the earliest possible point in the networking stack. When an attack is detected, XDP programs are dynamically loaded with filtering rules that drop malicious packets before they consume any significant system resources. Because XDP operates before sk_buff allocation, dropping packets at this layer has near-zero CPU cost -- enabling a single edge server to absorb millions of packets per second of attack traffic while continuing to serve legitimate requests.

The programmability of eBPF is key to Cloudflare's defense. Unlike static hardware-based DDoS mitigation, which can only filter on fixed header fields, eBPF programs can implement arbitrary packet inspection logic. When a novel attack pattern emerges, Cloudflare engineers can write and deploy a new XDP-based filter within minutes, without waiting for hardware vendors to update firmware or firmware-based rule sets.

Cloudflare has also pushed the boundaries of XDP performance optimization. Their flowtrackd system uses XDP to implement stateful connection tracking at the edge, distinguishing between established TCP connections (which should be allowed) and new connections that might be part of a SYN flood attack. This stateful processing in XDP -- traditionally considered impractical due to XDP's lack of sk_buff context -- demonstrates the creative engineering possible with eBPF.

Netflix: Network Performance at Scale

Netflix's use of eBPF focuses on network performance analysis and optimization across their massive streaming infrastructure. Netflix serves a substantial portion of global internet traffic, and even small improvements in network efficiency translate into significant cost savings and quality improvements.

Netflix's networking team uses eBPF-based tools to analyze TCP connection behavior at scale. Custom eBPF programs attached to TCP tracepoints collect per-connection metrics: congestion window evolution, retransmission rates, RTT measurements, and congestion events. This data is aggregated in-kernel and exported to Netflix's observability platform, providing fleet-wide visibility into network health that would be impossible to achieve with traditional packet capture or socket statistics.

One notable Netflix contribution is bpftrace scripts for diagnosing TCP performance anomalies. For example, their tcpretrans tool (part of the BCC toolkit, originally developed at Netflix) uses eBPF to trace every TCP retransmission event with the full connection context -- source, destination, TCP state, and congestion control algorithm. This enables rapid diagnosis of network path problems, misconfigured TCP parameters, and application-level issues that manifest as retransmissions.

Netflix has also explored eBPF for network traffic optimization, using custom XDP programs for traffic steering, ECMP (Equal-Cost Multi-Path) tuning, and congestion-aware routing decisions at the host level. These programs supplement the network fabric's routing with host-level intelligence, enabling finer-grained traffic management than is possible with network-level routing protocols alone.

Challenges and Considerations for Adoption

Despite its transformative potential, eBPF networking adoption comes with challenges that engineering teams must address.

Kernel Version Requirements

eBPF capabilities are tied to Linux kernel versions, and many of the advanced networking features require relatively recent kernels. XDP support varies by NIC driver and was only broadly available starting with kernel 4.18. Socket-level load balancing with cgroup/connect requires kernel 4.17 or later. BPF ring buffers require kernel 5.8. TCX (the modern tc attachment mechanism) requires kernel 6.6. Advanced features like BPF arena and kfuncs for networking are still landing in the latest kernel releases.

For organizations running older kernels (which is common in enterprise environments), upgrading to a kernel that supports the desired eBPF features may require significant testing and validation effort. Container-optimized OS distributions (like Bottlerocket, Flatcar, and Talos) tend to ship newer kernels and are well-suited for eBPF-based networking.

Debugging Complexity

When eBPF programs misbehave, debugging can be challenging. Traditional network debugging tools (tcpdump, ss, netstat) may not show the full picture because eBPF programs modify packet behavior before these tools see the traffic. Cilium provides cilium monitor and cilium bpf commands for inspecting eBPF program behavior, and Hubble provides network flow visibility, but the debugging experience is still more complex than traditional networking.

The eBPF verifier, while essential for safety, can produce cryptic error messages when rejecting programs. Understanding verifier errors requires knowledge of the eBPF instruction set, map types, and the specific constraints the verifier enforces. The community has improved verifier messages significantly in recent kernel versions, but the learning curve remains steep.

Multi-Cluster and Hybrid Environments

eBPF-based networking works best within a single cluster or across clusters connected by Cilium Cluster Mesh. In hybrid environments where some clusters use eBPF-based CNIs and others use traditional CNIs, the boundary between the two requires careful handling. Tunneling protocols (VXLAN, Geneve) bridge the gap but add latency and complexity.

Organizations planning eBPF networking adoption should develop a migration strategy that accounts for the transitional period where both eBPF and traditional networking coexist. Cilium supports a "chain" mode where it operates alongside an existing CNI, but this sacrifices some of the performance benefits of native eBPF networking.

Vendor Lock-In Considerations

Cilium's dominance in the eBPF networking space raises questions about vendor lock-in, particularly now that Cilium is backed by Isovalent (acquired by Cisco). While Cilium itself is open source (Apache 2.0 license) and part of the CNCF as a graduated project, the advanced features (Cilium Enterprise, Tetragon Enterprise) are commercial. Organizations should evaluate the boundary between open-source and commercial features and ensure that their core networking requirements are met by the open-source project.

Alternative eBPF-based CNIs exist (Calico's eBPF mode, for example), providing some protection against single-vendor dependency. The underlying eBPF kernel primitives are part of the Linux kernel and are not controlled by any single vendor.

Looking Ahead: The Future of eBPF Networking

The eBPF networking ecosystem continues to evolve rapidly. Several developments on the horizon will further expand what is possible:

BPF network namespace awareness is improving, allowing eBPF programs to operate across network namespace boundaries more seamlessly. This is critical for container networking, where each pod has its own network namespace, and the eBPF program on the host needs to understand the namespace context of each packet.

Hardware offload expansion beyond SmartNICs is bringing eBPF to DPUs (Data Processing Units) like NVIDIA BlueField-3 and AMD Pensando, which can run eBPF programs on dedicated ARM cores alongside the main host CPU. This enables line-rate packet processing with complex logic that exceeds what can be offloaded to SmartNIC hardware.

eBPF for Windows has reached a milestone where core XDP-like functionality is available on Windows Server, extending eBPF networking beyond Linux. While the Windows implementation is not yet at parity with Linux, it opens the door to cross-platform eBPF networking in heterogeneous environments.

L7 processing in eBPF continues to advance. Experimental work on HTTP/2 and gRPC parsing in eBPF programs points toward a future where the entire service mesh data plane, including application-layer routing, operates at kernel speed. Combined with kernel TLS offload, this could eliminate the need for userspace proxies entirely for a large class of workloads.

Integration with networking hardware is deepening. P4-programmable switches can work alongside eBPF programs on the host, creating a coordinated data plane that spans the network fabric and the endpoint. This convergence enables network-wide programmability that was previously impossible without proprietary hardware platforms.

Conclusion

eBPF has fundamentally transformed cloud-native networking. What was once an academic curiosity -- running sandboxed programs in the Linux kernel -- has become the foundation of networking infrastructure at the largest companies in the world and the default networking choice on every major cloud provider's managed Kubernetes platform.

The impact is measurable at every layer. XDP enables packet processing at tens of millions of packets per second per core, making software-based DDoS mitigation and load balancing competitive with dedicated hardware. tc BPF programs replace iptables with O(1) policy evaluation that scales to hundreds of thousands of services without degradation. Socket-level eBPF programs eliminate NAT overhead entirely for Kubernetes service traffic. The sidecar-less service mesh architecture reduces latency by an order of magnitude while reclaiming the memory consumed by per-pod proxy instances.

For networking and performance engineers, eBPF provides unprecedented visibility into production systems. Off-CPU analysis reveals hidden latency sources. Lock contention tracing identifies scalability bottlenecks. Network flow visibility with identity-aware metadata makes Kubernetes networking debuggable for the first time. And all of this instrumentation runs with overhead low enough for continuous production use.

The practical advice for organizations evaluating eBPF networking is straightforward: if you are running Kubernetes on any major cloud provider, eBPF-based networking is already the recommended default. Enable it. If you are running self-managed Kubernetes, evaluate Cilium as your CNI and kube-proxy replacement -- the performance benefits at scale are substantial. And if you have specialized networking requirements, invest in learning eBPF program development with libbpf or Aya -- the ability to write custom kernel-level networking programs is a capability that will define the next generation of network engineering.

The kernel is the network. eBPF made it programmable. The rest is engineering.