Day 15:AI 網絡瓶頸——由 iptables 轉向 eBPF
Day 15: AI Network Bottlenecks — Transitioning from iptables to eBPF
AI 工作負載下,Kubernetes 的 iptables 查找會按服務數量線性增加,消耗 CPU,亦會增加新連線延遲。eBPF 以核心雜湊表作固定時間查找,直接把封包送往目標 Pod,避免逐條檢查規則及額外經過用戶空間。遷移前應先量度主機 softirq CPU 時間及每條連線的延遲。
Week 3 turns to the network and security layer. Most teams tune the model and the GPU, then discover that the network path inside the cluster is eating CPU that inference needed. This lesson explains why, and what replaces it.
How a Service IP reaches a pod
When one pod calls a Kubernetes Service, kube-proxy translates the Service IP to a pod IP using Linux iptables rules. Rules are appended to chains, and the kernel checks a packet against them in sequence. That is an O(N) lookup. In a cluster with thousands of Services the evaluation adds measurable latency to every new connection, and burns host CPU. (kube-proxy also has an IPVS mode that uses hash tables, but many clusters still run the default iptables mode.)
Diagram: FIG 1 · With kube-proxy a packet is matched against iptables rules. An eBPF program in the kernel can map the destination directly, skipping those hops. See the animated version.
Diagram: FIG 2 · iptables evaluates rules in sequence, so cost grows with the number of services. A kernel hash map costs about the same however many services exist. The chart shows the shape, not measured numbers. See the animated version.
Why AI makes it worse
AI traffic is heavy. A request can carry a long context, and a RAG pipeline can stream batches of embeddings. A single 1,536-dimension float32 embedding is about 6 KB, a 128,000-token context is roughly half a megabyte of text, and a batch of 1,000 embeddings is around 6 MB. Multiply that by thousands of concurrent requests across replicas and vector databases, and packet handling competes with inference for the same CPUs.
Diagram: FIG 3 · Embeddings and long contexts are far larger than ordinary web requests. Sizes are approximate: 1,536 float32 values is about 6 KB, and 128,000 tokens is roughly half a megabyte of text. See the animated version.
eBPF: programs in the kernel
eBPF lets you load small, verified programs into the Linux kernel's execution path without writing a kernel module or rebooting. Instead of walking an iptables chain, an eBPF program on a kernel hook looks up the destination in a hash map, a constant-time operation that does not depend on how many Services exist, and sends the packet straight to the target pod. It also avoids extra trips through user space.
Diagram: FIG 4 · An eBPF program is checked by the kernel verifier before it runs, which makes it safe to load into the data path without a custom kernel module. See the animated version.
The safety comes from the verifier: before an eBPF program runs, the kernel checks that it cannot crash the kernel, access memory it should not, or loop forever. That is what makes it acceptable to run programs from a networking plugin inside the kernel data path.
What to take away
- The iptables model is O(N) in the number of services and can become a CPU tax for chatty AI workloads. Whether it hurts you depends on your cluster size and traffic, so measure before you migrate, for example by watching host CPU softirq time and per-connection latency.
- eBPF replaces the rule chain with a constant-time lookup, which scales with traffic instead of with service count.
- You do not write the eBPF programs yourself. A networking plugin such as Cilium ships them, and that is the next lesson.
You can see how many iptables rules a node carries with sudo iptables-save | wc -l. Compare that count on a small cluster and on a large one, and the growth with the number of Services becomes concrete.
Run it yourself
This is the exact example from the video. This counts steps for a rule chain and a hash lookup. It models the algorithms, it is not a benchmark of iptables or eBPF.
def chain(rules, key): # iptables: walk the rules in order
for n, r in enumerate(rules, 1):
if r == key:
return n
for n in (100, 1000, 10000):
rules = [f"svc-{i}" for i in range(n)]
table = {r: i for i, r in enumerate(rules)} # eBPF: a hash map
last = rules[-1]
table[last] # one lookup
print(f"{n:5d} svcs: chain {chain(rules, last):5d}, map 1")
Run it:
python scan.py
Output I got while recording the video:
100 svcs: chain 100, map 1
1000 svcs: chain 1000, map 1
10000 svcs: chain 10000, map 1
Coming Up Next
Day 16: Cilium and eBPF: sub-millisecond routing for heavy AI microservices.
eBPF #KubernetesNetworking #MLOps #LinuxKernel #PlatformEngineering #TechInfrastructure
Originally published at https://sureshpallapothu.in/blog/ai-int-day-15-ebpf-networking, where this post includes animated diagrams.
來源:DEV Community · dev.to