Pod Networking and CNI · Infra Illustrated

From Pod sandbox to network

Before a new Pod can send traffic, the runtime must give its sandbox an interface, an address, and routes. It asks the configured CNI plugin to perform that work. CNI defines the request and result exchanged at this boundary; the installed plugin decides what network topology and data plane provide the connection. Kubernetes network model ↗ CNI specification ↗

Lifecycle map

The runtime owns the CNI transaction

Kubernetes desired state reaches the plugin through the node runtime; the plugin then changes host and sandbox networking.

Control planePod scheduled→kubelet observes assignment

Node runtimeCreate sandbox→Invoke CNI ADD→Persist result

Network stateInterface + address→Routes / policy→Reachable Pod

Deletion reverses the transaction: runtime → CNI DEL → release attachments and IPAM state.

A real cluster may use veth pairs, Linux bridges, overlay tunnels, BGP routes, cloud network interfaces, eBPF programs, or Windows virtual switches. Kubernetes does not require one of these implementations, so identify the installed network plugin before interpreting host state.

The Pod sandbox

Every regular container in a Pod joins the same network namespace, or the equivalent isolation domain on its operating system. They see the same interfaces, addresses, routes, loopback device, and port space. One container can reach another through localhost, and two containers cannot bind the same address and port independently. Pod networking ↗

The runtime creates this sandbox before it attaches any network. Some runtimes keep the namespace alive with an infrastructure container commonly called “pause.” That container holds shared namespaces; it does not route Pod traffic. During diagnosis, inspect whether the sandbox and its network state exist instead of treating the pause container as the data plane.

hostNetwork: true is a deliberate exception: the Pod shares the Node network namespace. hostPort asks the network implementation to map a Node port toward a Pod port and can constrain scheduling. containerPort is descriptive metadata; it does not expose anything by itself.

The CNI transaction

The runtime invokes a CNI plugin with network configuration on standard input and request values such as CNI_COMMAND, CNI_CONTAINERID, CNI_NETNS, and CNI_IFNAME. For an ADD request, the plugin attaches the sandbox and returns the resulting interfaces, addresses, routes, and DNS information. The specification also defines cleanup, validation, chaining, delegation, and version negotiation. CNI execution protocol ↗

CNI lifecycle operations
Operation	Purpose	Failure implication
ADD	Attach a sandbox or modify its attachment	Pod sandbox creation normally fails or remains unready.
DEL	Undo attachment state and release resources	Leaked interfaces, routes, or IP reservations can affect later Pods.
CHECK	Compare existing state with the previous result	Can identify missing interfaces, routes, firewall state, or dependencies.
GC	Remove stale resources not associated with valid attachments	Complements cleanup but is not a substitute for DEL.
STATUS	Report whether a plugin can service new ADD requests	May reveal daemon failure or resource exhaustion.
VERSION	Negotiate supported specification versions	Runtime, config, and plugin compatibility must overlap.

When a configuration chains several plugins, ADD runs them in order and passes each result to the next plugin. DEL runs the chain in reverse. The primary plugin may attach an interface successfully and still have the complete transaction fail in a later port-mapping, bandwidth, or policy plugin.

IPAM and address ownership

IP address management may be built into the network implementation or delegated to another CNI plugin. The allocation source can be a node-local range, a cluster datastore, DHCP, a cloud subnet, or another provider-specific pool. “Every Node gets a /24” is one design, not a Kubernetes rule.

01Select an address familyIPv4, IPv6, or both must match cluster and plugin configuration.

02Reserve an addressThe IPAM implementation prevents concurrent reuse.

03Return routes and gatewayThe attachment result describes how the Pod leaves its namespace.

04Release on deletionDEL must return the allocation even if the namespace is already gone.

Address exhaustion follows the allocator’s real pool boundaries. One Node, zone, or cloud subnet can run out while another still has free addresses, so an apparently spacious cluster-wide Pod CIDR does not prove that this Node can attach another Pod. Inspect the allocator that owns the failing request.

Dual-stack requires compatible Nodes, pod networking, and control-plane configuration. A dual-stack Pod can receive one IPv4 and one IPv6 address, but application binding, policy, DNS answers, Services, and off-cluster routes must also support both families. Dual-stack requirements ↗

Same-node forwarding

On Linux, a common design connects the Pod namespace to the host using a veth pair. The host side may attach to a bridge, enter an L3 routing domain, or be managed by an eBPF data plane. Neighbor discovery, routes, and policy then determine how a packet reaches another local Pod.

Do not assume a bridge owns an “ARP table.” Hosts and Pods maintain neighbor tables; a Layer 2 bridge learns MAC forwarding information. Confusing the two leads to debugging the wrong cache.

Useful evidence includes the Pod’s addresses and routes, the host peer interface, neighbor state, policy counters, and a packet capture on both sides of the namespace boundary. Interface names are implementation-specific and can disappear when the sandbox is recreated.

Cross-node forwarding

For cross-node traffic, the network implementation must teach the source Node how to reach the destination Pod address. It may install an underlay route, wrap the packet in an overlay, or let the cloud network route the Pod address.

Packet path explorer“CNI” does not identify the path a packet takes.

Same Node

No underlay hop

Pod A netns

veth / hook

Node data plane

veth / hook

Pod B netns

10.244.1.7 → 10.244.1.9

The packet stays on one Node. A bridge, routes, or eBPF hooks may connect the two Pod interfaces.

Native routing

Pod packet on underlay

Pod A

Source Node route

Underlay network

Destination Node route

Pod B

10.244.1.7 → 10.244.2.8

The surrounding network learns routes to Pod prefixes. There is no tunnel header, but the underlay must route and permit Pod addresses.

Overlay

Inner + outer packet

Pod A

Encapsulate

Node-IP underlay

Decapsulate

Pod B

Inner 10.244.1.7 → 10.244.2.8 · Outer 192.0.2.10 → 192.0.2.20

Node-to-node encapsulation carries the original Pod packet. Tunnel headers consume MTU and require an intact return path.

Cloud-native

Provider-routed Pod address

Pod A

Pod-facing interface

Cloud fabric

Destination interface

Pod B

Provider-routable Pod IP → provider-routable Pod IP

Pod addressing integrates with cloud networking. Subnet capacity, interface quotas, security controls, and provider routes become part of Pod networking.

eBPF is a data-plane mechanism, not a fifth topology. An eBPF implementation can program same-node, native, overlay, or cloud-integrated paths.

Identify the outer source and destination, any encapsulation, who owns the route, and how the reply returns. Those details come from the installed network implementation rather than the CNI specification.

MTU and fragmentation

Encapsulation consumes bytes. If the Pod interface MTU is copied from the Node interface without subtracting tunnel overhead, small requests can succeed while larger transfers stall. Path MTU discovery also depends on relevant ICMP or ICMPv6 messages reaching the sender.

Compare the application payload path, Pod interface MTU, tunnel interface MTU, Node underlay MTU, and any cloud or VPN segment. Avoid hard-coding a universal overlay deduction: headers vary by IP family, encapsulation, encryption, and implementation.

Egress and masquerading

The no-NAT Pod-to-Pod model does not mean Pod traffic is never translated. Off-cluster destinations often cannot route return traffic to Pod CIDRs, so an egress data plane may masquerade the Pod source as a Node, gateway, or other routable address. This affects logs, allowlists, policy, and return routing.

Ask separately whether the destination is inside the pod network, another cluster, a private routed network, or the public internet. Each boundary may select a different route, NAT rule, gateway, or policy layer.

Policy is adjacent, not identical

NetworkPolicy is a Kubernetes API generally enforced by the pod network implementation, but basic connectivity and policy enforcement are distinct capabilities. A plugin can provide Pod connectivity without implementing NetworkPolicy; the API objects then exist but have no effect. Network plugin responsibilities ↗

Policy programming can occur at veth hooks, host firewall tables, eBPF hooks, virtual switches, or provider controls. Inspecting only iptables is insufficient when the implementation uses another enforcement plane.

Diagnose sandbox failures

Read Pod events and container-runtime errors. Messages around sandbox creation and CNI ADD are more useful than repeated application restarts.

Confirm the network plugin agents are ready on the affected Node and inspect their logs, configuration, credentials, and datastore connectivity.

Compare a healthy Pod on the same Node, a healthy Pod on another Node, and the failing Pod. This separates node-local attachment from cluster-wide routing.

Inspect Pod and Node addresses, routes, interfaces, neighbor tables, policy state, and IPAM capacity using tools appropriate to the implementation.

Test same-node and cross-node Pod IPs in both directions. A one-way result usually points toward return routing, asymmetric policy, or translation.

For size-dependent failures, test progressively larger payloads and capture ICMP/ICMPv6 alongside tunnel traffic before changing MTUs.

What this means in practice

Choose a Pod network by matching it to the address scale, underlay, policy model, operating systems, encryption, observability, performance, and skills your cluster requires. A generic throughput benchmark cannot answer those design questions. Before an upgrade, verify compatibility across Kubernetes, the runtime, CNI version, plugin release, kernel, and managed-service constraints.

Keep a cluster-specific runbook recording the real Pod CIDRs, IPAM owner, forwarding mode, encapsulation, MTU, egress translation, policy engine, kube-proxy replacement status, and diagnostic commands. That inventory turns “the network is broken” into a bounded investigation.

Authoritative references