Field Manual
K8s & Docker Networking
NET-K8S-001
INDEX
NET
Container Networking Handbook

Networking
for K8s
& Docker

// "Every pod gets an IP. Every service gets a name. Nothing talks without a policy."

From subnets and OSI layers to Ingress controllers, CoreDNS, service meshes, and zero-trust network policies. A complete field guide for DevOps engineers, beginner to intermediate.

Docker Kubernetes CNI Ingress / Egress Service Mesh Zero Trust
01

Basic Networking Concepts

// THE VOCABULARY YOU NEED BEFORE ANYTHING ELSE

Before diving into Docker or Kubernetes, you need a firm grip on the building blocks. Container networking is just regular networking under a layer of abstraction — understanding the fundamentals prevents mystery when things break.

IP Address

A unique numerical label assigned to every device (or container) on a network. IPv4 uses 32-bit addresses (e.g. 192.168.1.10). IPv6 uses 128-bit addresses (e.g. fd00::1). Every pod in Kubernetes gets its own IP.

Port

A number (1–65535) that identifies a specific process on a host. Think of the IP as the building address and the port as the apartment number. 80 = HTTP, 443 = HTTPS, 8080 = common app port.

Protocol

Rules for how data is formatted and transmitted. TCP — reliable, ordered, connection-based (web, APIs). UDP — fast, connectionless, no delivery guarantee (DNS queries, video streaming).

Subnetting & CIDR

A subnet divides a larger network into smaller segments. CIDR (Classless Inter-Domain Routing) notation expresses an IP range compactly. This is the notation Kubernetes uses everywhere for pod and service CIDRs.

CIDR NotationSubnet MaskTotal IPsUsable HostsTypical Use
/8255.0.0.016,777,21616,777,214Large cloud VPCs
/16255.255.0.065,53665,534Kubernetes cluster CIDR
/24255.255.255.0256254Single node subnet, Docker bridge
/28255.255.255.2401614Small service subnet
/32255.255.255.25511Single host / pod IP
▸
K8s defaults you'll see constantly: Pod CIDR 10.244.0.0/16 (Flannel), Service CIDR 10.96.0.0/12 (default kubeadm). These are private RFC 1918 ranges — they never appear on the public internet.

Ports & Common Protocols

22
TCP
SSH — remote shell access
80
TCP
HTTP — plain web traffic
443
TCP
HTTPS — encrypted web traffic
53
UDP/TCP
DNS — name resolution
6443
TCP
Kubernetes API server
2379-80
TCP
etcd — K8s state store
10250
TCP
kubelet API
30000-32767
TCP
K8s NodePort range
8080
TCP
Common app / API port
5432
TCP
PostgreSQL
6379
TCP
Redis cache
9092
TCP
Kafka broker
02

OSI & TCP/IP Models

// WHERE IN THE STACK DOES IT LIVE?

The OSI model is a conceptual framework that describes how data travels across a network in 7 layers. When someone says "Layer 4 load balancer" or "Layer 7 routing" — this is what they mean. Kubernetes Ingress operates at Layer 7. Services operate at Layer 4.

7
Application
HTTP, HTTPS, gRPC, WebSocket
Ingress rules, API routing
6
Presentation
TLS/SSL encryption, encoding
TLS termination at Ingress
5
Session
Session management, auth tokens
Connection pooling
4
Transport
TCP, UDP — ports & connections
K8s Services, NodePort, LB
3
Network
IP, ICMP — routing & addressing
Pod IPs, CNI plugins, iptables
2
Data Link
MAC addresses, Ethernet frames
Docker bridge (veth pairs)
1
Physical
Cables, NIC, signal transmission
The actual host network
ℹ
Practical tip: When Kubernetes creates a pod, it assigns a virtual ethernet pair (veth) — one end inside the pod's network namespace, one end on the host. This operates at Layer 2 (Data Link). The CNI plugin then configures Layer 3 routing so pods on different nodes can reach each other.

TCP vs UDP — When to Use What

TCP — Reliable
  • Connection-oriented (3-way handshake)
  • Guaranteed delivery, ordered packets
  • Error detection and retransmission
  • Slower due to overhead
  • Use for: HTTP/S, gRPC, databases, APIs
UDP — Fast
  • Connectionless — fire and forget
  • No delivery guarantee, no order
  • Minimal overhead, very fast
  • Lost packets are NOT retransmitted
  • Use for: DNS, video streams, game data
03

IP Addressing, Subnets & DNS

// THE PHONE BOOK OF THE INTERNET

DNS (Domain Name System) translates human-readable names like api.myapp.com into IP addresses. In Kubernetes, CoreDNS does this for every service — so instead of remembering 10.96.45.12, you write user-service.production.svc.cluster.local.

DNS Resolution Flow
  • App calls db.default.svc.cluster.local
  • Query goes to CoreDNS (ClusterIP: 10.96.0.10)
  • CoreDNS returns the Service ClusterIP
  • Traffic routes to one of the pods behind the Service
  • kube-proxy handles the final NAT/iptables routing
Private IP Ranges (RFC 1918)
  • 10.0.0.0/8 — largest private block (Kubernetes pods)
  • 172.16.0.0/12 — Docker's default bridge range
  • 192.168.0.0/16 — home/office routers
  • These never route on the public internet
  • All container networks use these ranges

NAT — Network Address Translation

NAT allows many private IP addresses to share one public IP. Docker uses NAT to let containers reach the internet via the host's public IP. Kubernetes uses SNAT (Source NAT) when pods reach external services. iptables rules handle this transparently.

bash — View iptables NAT rules (what K8s manages)
# View K8s service NAT chains sudo iptables -t nat -L KUBE-SERVICES --line-numbers # See how a ClusterIP maps to pod IPs sudo iptables -t nat -L KUBE-SVC-XXXXXXXXXXX -n -v # Docker's NAT masquerade rule (containers → internet) sudo iptables -t nat -L POSTROUTING -v | grep masquerade
04

Docker Network Overview

// HOW CONTAINERS GET CONNECTIVITY

When Docker starts, it creates a virtual network called docker0 on the host — a software bridge operating at Layer 2. Every container gets a virtual ethernet (veth) pair: one end inside the container's private network namespace, the other attached to docker0. Containers on the same bridge can talk to each other; traffic to the host or internet is NAT'd.

HOST MACHINE
🌐
Public
eth0 / NIC
│ NAT / iptables
🌉
Bridge
docker0
│ veth0
│ veth1
│ veth2
📦
Container
frontend
172.17.0.2
⟷
📦
Container
api
172.17.0.3
⟷
📦
Container
postgres
172.17.0.4
⚠
Default bridge network limitation: Containers on the default bridge network can only communicate via IP address — no DNS name resolution between them. Always create a user-defined bridge network (or use Docker Compose) to get automatic DNS by container name.
05

Docker Network Drivers

// PICKING THE RIGHT DRIVER FOR THE JOB
bridge DEFAULT

Software bridge on the host. Containers on the same bridge can communicate. Requires port mapping to reach from outside. Use for: local dev, single-host setups. DNS by container name works on user-defined bridges.

host PERFORMANCE

Container shares the host's network stack directly — no isolation. The container binds ports on the host directly. Use for: performance-critical apps where network overhead matters. No port mapping needed.

overlay MULTI-HOST

Spans multiple Docker hosts (Docker Swarm mode). Encapsulates packets using VXLAN to create a virtual L2 network across hosts. Use for: Swarm multi-node deployments. Not commonly used with Kubernetes.

macvlan LEGACY

Gives each container its own MAC address, making it appear as a physical device on the network. Use for: legacy apps that require a real NIC. Complex to configure.

none ISOLATED

No network interface configured. Container is completely network-isolated. Use for: batch jobs, security-sensitive workloads, or when networking is handled externally.

ipvlan ADVANCED

Similar to macvlan but shares the parent MAC address. Useful when the switch limits MAC address counts. Use for: data center environments with strict MAC policies.

bash — Docker network commands
# List all networks docker network ls # Create a user-defined bridge (enables DNS by container name) docker network create --driver bridge my-app-net # Run containers on the same network (they can ping each other by name) docker run -d --name api --network my-app-net myapi:latest docker run -d --name frontend --network my-app-net myfrontend:latest # From 'frontend' container, reach 'api' by name: # curl http://api:8080/health ← works! DNS resolves 'api' to its container IP # Inspect a network (see connected containers, IPs) docker network inspect my-app-net # Connect a running container to a network docker network connect my-app-net my-existing-container # Host network mode — container uses host's network stack docker run --network host nginx
06

Docker Compose & Service Communication

// MULTI-CONTAINER APPS MADE EASY

Docker Compose automatically creates a user-defined bridge network for your project and connects all services to it. Services can reach each other using their service name as the hostname. This is the foundation of multi-container communication on a single host.

yaml — docker-compose.yml (full stack example)
version: '3.9' services: frontend: image: nginx:alpine ports: - "80:80" # host:container — expose to outside world networks: - frontend-net depends_on: - api api: build: ./api ports: - "8080:8080" # optional: expose for local dev debugging environment: - DB_HOST=postgres # ← use service name, not IP! - DB_PORT=5432 - REDIS_URL=redis://cache:6379 networks: - frontend-net # reachable by frontend - backend-net # reachable by backend services depends_on: - postgres - cache worker: build: ./worker environment: - DB_HOST=postgres - KAFKA_BROKERS=kafka:9092 networks: - backend-net # isolated — NOT on frontend-net postgres: image: postgres:15 volumes: - pg-data:/var/lib/postgresql/data environment: POSTGRES_PASSWORD: secret networks: - backend-net # No 'ports' here = NOT accessible from host (security best practice) cache: image: redis:7-alpine networks: - backend-net networks: frontend-net: driver: bridge # frontend + api can talk backend-net: driver: bridge # api + worker + postgres + redis can talk # postgres is isolated from frontend — good security! volumes: pg-data:
▸
Security pattern above: The database (postgres) is only on backend-net — not on frontend-net. This means even if the frontend is compromised, it cannot directly reach the database. The API bridges both networks and acts as the only authorized caller.

How Service Name DNS Works in Compose

Service
api container
→
Calls
postgres:5432
→
Docker DNS
Resolves "postgres"
→
Returns IP
172.20.0.3
→
TCP
Connect to DB
07

Port Mapping & EXPOSE

// CONTROLLING WHAT'S REACHABLE FROM OUTSIDE
EXPOSE (Dockerfile)

EXPOSE 8080 in a Dockerfile is documentation only. It tells other developers which port the app listens on, but does NOT actually publish it. Think of it as a label, not a firewall rule.

Dockerfile
FROM node:20-alpine EXPOSE 3000 # documentation only CMD ["node", "server.js"]
-p / ports (Runtime)

-p 8080:3000 actually publishes the port. Format: HOST_PORT:CONTAINER_PORT. Traffic hitting the host on port 8080 is forwarded to port 3000 inside the container via iptables DNAT rules.

bash
# Publish port: host 8080 → container 3000 docker run -p 8080:3000 myapp # Bind only to localhost (secure) docker run -p 127.0.0.1:8080:3000 myapp # Random host port (Docker assigns) docker run -p 3000 myapp
⚠
Docker bypasses UFW/firewalld: Docker modifies iptables directly. Even if you block a port in UFW, if Docker published it, it may still be accessible. Always bind to 127.0.0.1 for local-only services: -p 127.0.0.1:5432:5432.
08

Kubernetes Network Model

// THE FOUR FUNDAMENTAL RULES

Kubernetes networking is built on a set of fundamental requirements that every CNI plugin must implement. Understanding these rules is key to reasoning about why things work the way they do.

Rule 1 — Every Pod Gets a Unique IP

Every pod in the cluster gets its own IP address from the pod CIDR range. You never need to map ports between containers in the same pod — they share a network namespace and communicate via localhost.

Rule 2 — Pods Can Reach Any Pod Directly

Any pod can communicate with any other pod in the cluster using its pod IP — no NAT, no port mappings. This flat network model is enforced by the CNI plugin across nodes.

Rule 3 — No IP Masquerading Between Pods

When pod A talks to pod B, the source IP seen by pod B is the actual IP of pod A — not NAT'd. This matters for logging, security, and network policies that match on source IP.

Rule 4 — Agents Can Reach All Pods

Node-level agents (kubelet, kube-proxy, monitoring daemons) can reach all pods on all nodes. This enables health checks, metrics scraping, and log collection.

Pod-to-Pod Communication Across Nodes

Node 1
Pod A
10.244.0.5
→
veth pair
eth0 → cbr0
→
CNI
Routing / Encap
→
Node 2
Decap + Route
→
Node 2
Pod B
10.244.1.7
09

Pod Networking

// CONTAINERS THAT SHARE A NETWORK NAMESPACE

A Kubernetes Pod is a group of one or more containers that share a network namespace — meaning they share the same IP address, the same set of ports, and the same localhost. They communicate with each other via localhost, just like processes on the same machine.

yaml — Multi-container Pod (sidecar pattern)
apiVersion: v1 kind: Pod metadata: name: app-with-sidecar spec: containers: - name: api image: myapi:v1.2 ports: - containerPort: 8080 # This container listens on :8080 - name: log-collector image: fluentd:v1.16 # This container can call localhost:8080 to scrape metrics # They share the same network namespace — no service needed! env: - name: API_ENDPOINT value: "http://localhost:8080/metrics" - name: envoy-proxy image: envoyproxy/envoy:v1.28 ports: - containerPort: 9901 # admin port - containerPort: 15090 # Prometheus metrics # Intercepts all traffic via iptables before it reaches 'api'
pause / infra container

Every pod starts an invisible "pause" container (also called infra or sandbox container). Its job is to hold the network namespace open. Even if app containers restart, the pod IP stays the same because the pause container never stops.

Container Port ≠ Published Port

containerPort in a Pod spec is also documentation-only, like Docker's EXPOSE. Traffic is only routed to a pod when a Service selects it and routes to the specified port. The containerPort field helps humans and tooling understand what's running.

10

Kubernetes Services

// STABLE ENDPOINTS FOR UNSTABLE PODS

Pods are ephemeral — they die and are replaced with new IPs constantly. A Service gives you a stable virtual IP (ClusterIP) and a DNS name that always points to the healthy pods behind it, regardless of pod restarts or scaling events. kube-proxy maintains iptables/IPVS rules to route traffic to the right pods.

Service TypeAccessible FromHow it WorksUse Case
ClusterIP Inside cluster only Virtual IP only routable within the cluster. Default type. Internal microservice comms (API → DB)
NodePort Outside + Inside Opens a static port (30000–32767) on every node. Traffic on NodeIP:30080 → pods. Dev/test, no cloud LB available
LoadBalancer External internet Cloud provider provisions an external LB with a public IP. Routes to NodePort → pods. Production external access
ExternalName Inside cluster only DNS CNAME alias to an external hostname. No proxy, no pods. Map external service into cluster DNS
Headless Inside cluster only ClusterIP: None. DNS returns individual pod IPs (not a VIP). Used with StatefulSets. Stateful apps: Cassandra, Kafka, MySQL
yaml — All Service Types
## 1. ClusterIP — internal only (most common) apiVersion: v1 kind: Service metadata: name: api-service namespace: production spec: selector: app: api # routes to pods with this label ports: - protocol: TCP port: 80 # port OTHER services call targetPort: 8080 # port the pod actually listens on type: ClusterIP # default — omit this line and you get ClusterIP --- ## 2. NodePort — exposed on every node apiVersion: v1 kind: Service metadata: name: api-nodeport spec: selector: app: api ports: - port: 80 targetPort: 8080 nodePort: 30080 # access via NodeIP:30080 from outside type: NodePort --- ## 3. LoadBalancer — cloud-provisioned external IP apiVersion: v1 kind: Service metadata: name: api-lb annotations: service.beta.kubernetes.io/aws-load-balancer-type: "nlb" spec: selector: app: api ports: - port: 443 targetPort: 8080 type: LoadBalancer --- ## 4. Headless — for StatefulSets (DNS returns pod IPs) apiVersion: v1 kind: Service metadata: name: postgres-headless spec: clusterIP: None # headless! selector: app: postgres ports: - port: 5432
11

CoreDNS & Service Discovery

// HOW PODS FIND EACH OTHER BY NAME

CoreDNS runs as a Deployment inside Kubernetes and acts as the cluster's DNS server. Every pod is configured (via /etc/resolv.conf) to send DNS queries to the CoreDNS ClusterIP. CoreDNS resolves Service names and returns their ClusterIP addresses.

DNS Name Formats

What you're queryingShort form (same namespace)Full FQDN
Service in same namespace api-service api-service.production.svc.cluster.local
Service in different namespace api-service.production api-service.production.svc.cluster.local
Headless pod (StatefulSet) N/A pod-0.postgres.production.svc.cluster.local
External domain (forwarded) google.com Forwarded to upstream DNS resolver
bash — DNS debugging inside pods
# Run a debug pod with DNS tools kubectl run dns-debug --image=busybox:1.35 --rm -it -- sh # Inside pod: check what DNS server is configured cat /etc/resolv.conf # nameserver 10.96.0.10 ← CoreDNS ClusterIP # search production.svc.cluster.local svc.cluster.local cluster.local # Resolve a service — short form works because of search domains nslookup api-service nslookup api-service.production.svc.cluster.local # Full DNS lookup with dig kubectl run debug --image=dnsutils --rm -it -- dig api-service.production.svc.cluster.local # Check CoreDNS logs kubectl logs -n kube-system -l k8s-app=kube-dns --tail=50 # Verify CoreDNS ConfigMap (forward rules, custom zones) kubectl get configmap coredns -n kube-system -o yaml
12

CNI Plugins

// THE NETWORK PLUMBERS OF KUBERNETES

CNI (Container Network Interface) is the plugin standard that Kubernetes uses to set up pod networking. When a pod is created, Kubernetes calls the CNI plugin to assign an IP, configure the veth pair, and set up routing. Without a CNI plugin, pods cannot communicate.

CNI PluginModelNetwork PolicyPerformanceBest For
Flannel VXLAN overlay No Medium Simplest setup, dev/test clusters
Calico BGP routing or overlay Yes High Production, large clusters, network policies
Cilium eBPF-based Yes (L7) Highest Security-focused, observability, service mesh lite
Weave Overlay mesh Yes Medium Simple multi-cloud / multi-host
AWS VPC CNI Native VPC IPs Via SGs High EKS — pods get real VPC IPs
▸
Cilium with eBPF replaces iptables with eBPF programs loaded directly into the Linux kernel. This eliminates the O(n) cost of iptables rule evaluation (critical at scale) and enables HTTP/gRPC-aware network policies — e.g., "only allow GET /api/v1/users, deny all other HTTP paths."
13

Ingress Controllers

// THE FRONT DOOR OF YOUR CLUSTER

An Ingress is a Kubernetes resource that defines HTTP/HTTPS routing rules — which hostnames and URL paths map to which internal Services. An Ingress Controller (nginx, Traefik, HAProxy, or a cloud-native one) is the actual reverse proxy that reads those rules and routes traffic. Think of Ingress as the config and the controller as the engine.

External Traffic Flow
🌍
Internet
User Browser
→
⚖️
Cloud LB
External IP
:443
→
🚪
Ingress
nginx controller
TLS termination
↓ route by host/path
🎨
ClusterIP
frontend-svc
:80
|
⚙️
ClusterIP
api-svc
:8080
|
📊
ClusterIP
admin-svc
:3000
yaml — Ingress resource (nginx)
apiVersion: networking.k8s.io/v1 kind: Ingress metadata: name: app-ingress namespace: production annotations: kubernetes.io/ingress.class: "nginx" nginx.ingress.kubernetes.io/ssl-redirect: "true" nginx.ingress.kubernetes.io/rate-limit-rpm: "100" cert-manager.io/cluster-issuer: "letsencrypt-prod" spec: tls: - hosts: - myapp.com - api.myapp.com secretName: myapp-tls-secret # cert-manager auto-fills this rules: # Route 1: main website - host: myapp.com http: paths: - path: / pathType: Prefix backend: service: name: frontend-svc port: number: 80 # Route 2: API subdomain - host: api.myapp.com http: paths: - path: /v1 # /v1/* → api-service pathType: Prefix backend: service: name: api-svc port: number: 8080 - path: /health # exact match pathType: Exact backend: service: name: api-svc port: number: 8080

Popular Ingress Controllers

NGINX Ingress MOST COMMON

Community + official Kubernetes version. Extensive annotation-based configuration. Handles SSL termination, rate limiting, auth, rewrites. Best for teams familiar with nginx config.

Traefik CLOUD NATIVE

Automatic service discovery — reads K8s resources and auto-configures routes. Native Let's Encrypt integration. Built-in dashboard. Popular in homelab and smaller production setups.

AWS ALB / GCP GLB MANAGED

Cloud provider manages the load balancer. aws-load-balancer-controller provisions ALBs automatically from Ingress resources. Best for EKS/GKE — no nginx to manage.

ℹ
Ingress vs Gateway API: The Gateway API is the next-generation replacement for Ingress, with richer routing primitives (HTTPRoute, GRPCRoute, TCPRoute). It's now GA in Kubernetes 1.28+. Ingress still works perfectly and is widely supported, but new projects should evaluate Gateway API for its expressiveness and role-based access model.
14

Egress & Network Policy

// CONTROLLING WHAT TRAFFIC LEAVES YOUR PODS

Egress refers to outbound traffic from pods — pods talking to the internet, to external databases, or to other services. By default, Kubernetes allows all egress. Network Policies let you restrict which pods can talk to what, in both directions.

Default Behaviour (No Policies Applied)

No NetworkPolicy = No Isolation
  • Any pod can reach any other pod in the cluster
  • Any pod can reach external internet endpoints
  • Any pod can reach the Kubernetes API server
  • This is a significant security risk in production
  • A compromised pod can pivot to reach all other services
With NetworkPolicy = Explicit Allow
  • Default deny-all once ANY policy selects a pod
  • Only explicitly listed sources/destinations allowed
  • Ingress rules: who can send traffic TO this pod
  • Egress rules: where this pod can SEND traffic TO
  • CNI plugin must support NetworkPolicy (Calico, Cilium)
yaml — Network Policies (Ingress + Egress)
## 1. Default deny-all (apply to every namespace as baseline) apiVersion: networking.k8s.io/v1 kind: NetworkPolicy metadata: name: default-deny-all namespace: production spec: podSelector: {} # selects ALL pods in this namespace policyTypes: - Ingress - Egress # No rules = deny everything --- ## 2. Allow API pods to receive from frontend + send to DB apiVersion: networking.k8s.io/v1 kind: NetworkPolicy metadata: name: api-policy namespace: production spec: podSelector: matchLabels: app: api # applies to pods labelled app=api policyTypes: - Ingress - Egress ingress: - from: - podSelector: matchLabels: app: frontend # only frontend can call api - podSelector: matchLabels: app: ingress-nginx # and the ingress controller ports: - protocol: TCP port: 8080 egress: - to: - podSelector: matchLabels: app: postgres # api → postgres only ports: - protocol: TCP port: 5432 - to: - podSelector: matchLabels: app: redis ports: - protocol: TCP port: 6379 - to: # allow DNS (required!) - namespaceSelector: matchLabels: kubernetes.io/metadata.name: kube-system ports: - protocol: UDP port: 53 --- ## 3. Allow egress to specific external IP (e.g., external API) apiVersion: networking.k8s.io/v1 kind: NetworkPolicy metadata: name: payment-external-egress spec: podSelector: matchLabels: app: payment-service policyTypes: - Egress egress: - to: - ipBlock: cidr: 52.94.76.0/24 # Stripe API IP range ports: - protocol: TCP port: 443
⚠
DNS egress is easy to forget! When you apply a NetworkPolicy with egress rules, DNS queries (port 53 to CoreDNS) are also blocked by default. Your pod will fail to resolve any name. Always explicitly allow egress to kube-system namespace on port 53/UDP as shown above.
15

Frontend ↔ Backend Communication

// THE COMPLETE PICTURE OF A WEB REQUEST

Let's trace a full request from a user's browser through the cluster to the database and back. This is the pattern you'll implement in almost every production application.

Browser
GET /api/products
→
DNS
api.myapp.com → 1.2.3.4
→
Cloud LB
TCP :443
→
Ingress
TLS term + route
→
api-svc
ClusterIP:8080
→
api pod
Process request
→
db-svc
postgres:5432

Frontend Served from K8s vs CDN

Pattern A: API-First SPA COMMON
  • React/Vue/Angular built to static files
  • Served from CDN (Cloudflare, CloudFront) or nginx pod
  • App makes API calls to api.myapp.com
  • CORS must be configured on the backend
  • Ingress routes /api/* to backend, /* to frontend
Pattern B: Backend-for-Frontend RECOMMENDED
  • Dedicated BFF service per client type (web, mobile)
  • BFF aggregates calls to multiple microservices
  • Browser only talks to BFF (single origin — no CORS)
  • BFF handles auth session, token exchange
  • Each BFF is a ClusterIP service + Ingress rule

CORS in Kubernetes Context

yaml — nginx Ingress CORS annotation
apiVersion: networking.k8s.io/v1 kind: Ingress metadata: name: api-ingress annotations: nginx.ingress.kubernetes.io/enable-cors: "true" nginx.ingress.kubernetes.io/cors-allow-origin: "https://myapp.com" nginx.ingress.kubernetes.io/cors-allow-methods: "GET, POST, PUT, DELETE, OPTIONS" nginx.ingress.kubernetes.io/cors-allow-headers: "Authorization, Content-Type" nginx.ingress.kubernetes.io/cors-allow-credentials: "true" spec: rules: - host: api.myapp.com http: paths: - path: / pathType: Prefix backend: service: name: api-svc port: number: 8080

WebSocket Support

yaml — Ingress with WebSocket (sticky sessions)
apiVersion: networking.k8s.io/v1 kind: Ingress metadata: name: ws-ingress annotations: nginx.ingress.kubernetes.io/proxy-read-timeout: "3600" nginx.ingress.kubernetes.io/proxy-send-timeout: "3600" # WebSocket requires long-lived connections — increase timeouts nginx.ingress.kubernetes.io/upstream-hash-by: "$request_uri" # Sticky sessions by URI — WebSocket must reach same pod spec: rules: - host: ws.myapp.com http: paths: - path: /ws pathType: Prefix backend: service: name: realtime-svc port: number: 3001
16

Microservices Communication

// SYNC, ASYNC, AND EVERYTHING IN BETWEEN

In a microservices architecture, services need to talk to each other. There are two broad categories: synchronous (the caller waits for the response) and asynchronous (the caller publishes an event and continues). The right choice depends on your latency, reliability, and coupling requirements.

Synchronous Communication (REST / gRPC)

REST over HTTP STANDARD
  • Service calls http://order-svc:8080/v1/orders/123
  • K8s Service DNS resolves order-svc to ClusterIP
  • Simple, universally understood, easy to debug
  • Drawback: tight coupling — if order-svc is down, caller fails
  • Implement circuit breakers and retries
gRPC HIGH PERFORMANCE
  • Binary protocol (Protocol Buffers) — 3-5× smaller payload than JSON
  • HTTP/2 multiplexing — multiple streams per connection
  • Strongly typed contracts via .proto files
  • Requires L7 load balancing for proper round-robin
  • Best for: internal service-to-service high-throughput calls
yaml — Headless Service for gRPC (enables client-side LB)
# gRPC with a regular ClusterIP won't distribute connections well # because HTTP/2 uses a single long-lived connection. # Use a headless Service so clients get ALL pod IPs and do their own LB: apiVersion: v1 kind: Service metadata: name: grpc-order-svc spec: clusterIP: None # headless — DNS returns all pod IPs selector: app: order-service ports: - name: grpc port: 50051 targetPort: 50051 # gRPC client resolves DNS and gets: [10.244.1.5, 10.244.2.8, 10.244.3.2] # Client-side load balancer (e.g., grpc.Dial with round_robin) distributes calls

Asynchronous Communication (Event-Driven)

Message Queues (RabbitMQ / SQS) DECOUPLED
  • Producer puts message in queue, continues immediately
  • Consumer processes at its own pace
  • If consumer is down, messages queue up (not lost)
  • One-to-one: each message consumed by exactly one consumer
  • Use for: order processing, email sending, file uploads
Event Streaming (Kafka) SCALABLE
  • Events stored durably on topics (like a distributed log)
  • Multiple consumers can read the same event independently
  • Replay past events (not possible with queues)
  • Consumer groups enable horizontal scaling
  • Use for: audit logs, analytics, event sourcing
yaml — Kafka in K8s (service discovery pattern)
# Kafka StatefulSet needs a headless service for pod-to-pod comms # and a regular service for external client access ## Headless service — brokers discover each other via DNS apiVersion: v1 kind: Service metadata: name: kafka-headless spec: clusterIP: None selector: app: kafka ports: - name: broker port: 9092 ## Client-facing service — apps connect to this apiVersion: v1 kind: Service metadata: name: kafka spec: selector: app: kafka ports: - port: 9092 # App config: KAFKA_BROKERS=kafka.production.svc.cluster.local:9092 # Kafka broker DNS: kafka-0.kafka-headless.production.svc.cluster.local # kafka-1.kafka-headless.production.svc.cluster.local

Microservice Communication Patterns

PatternProtocolCouplingLatencyWhen to Use
Direct RESTHTTP/1.1TightLowSimple CRUD, reads, user-facing actions
gRPCHTTP/2 + ProtobufTightVery LowHigh-throughput internal calls, streaming
Message QueueAMQP / SQSLooseMediumBackground jobs, notifications, reliability needed
Event StreamKafka / KinesisVery LooseMediumAudit, analytics, fan-out, event sourcing
GraphQLHTTPTightLowBFF pattern, flexible client queries
Service MeshmTLS / EnvoyManaged+~1ms overheadSecurity, observability, traffic management
ℹ
Saga Pattern for distributed transactions: When an action spans multiple services (e.g., place order → reserve inventory → charge payment → send email), use the Saga pattern. Each service performs its local transaction and publishes an event. If any step fails, compensating transactions roll back previous steps. This avoids distributed ACID transactions across services.
17

Service Mesh

// NETWORK INTELLIGENCE WITHOUT CODE CHANGES

A service mesh injects a lightweight proxy sidecar (Envoy) into every pod. All traffic passes through the sidecar — giving you mTLS encryption, retries, circuit breaking, distributed tracing, and traffic splitting without changing a single line of application code.

Without Service Mesh
  • Plain HTTP between services (unencrypted)
  • No automatic retries on failure
  • No circuit breaker — cascading failures
  • Tracing requires SDK in every service
  • Traffic splitting requires code changes
  • No mutual authentication between services
With Istio / Linkerd
  • Automatic mTLS on all service-to-service traffic
  • Configurable retry policies (3 retries, 100ms backoff)
  • Circuit breaker built into Envoy proxy
  • Traces injected at proxy level — zero SDK needed
  • Canary: 90% traffic v1, 10% traffic v2 via config
  • Service identity via SPIFFE/SVID certificates
yaml — Istio Traffic Management (Canary Deploy)
## VirtualService — traffic routing rules apiVersion: networking.istio.io/v1beta1 kind: VirtualService metadata: name: api-traffic-split spec: hosts: - api-svc http: - match: - headers: x-canary: exact: "true" # canary header → always v2 route: - destination: host: api-svc subset: v2 - route: # default: 90/10 split - destination: host: api-svc subset: v1 weight: 90 - destination: host: api-svc subset: v2 weight: 10 --- ## DestinationRule — defines subsets (v1 / v2) apiVersion: networking.istio.io/v1beta1 kind: DestinationRule metadata: name: api-destination spec: host: api-svc trafficPolicy: connectionPool: tcp: maxConnections: 100 outlierDetection: consecutive5xxErrors: 5 # circuit breaker: eject after 5 errors interval: 30s baseEjectionTime: 30s subsets: - name: v1 labels: version: v1 - name: v2 labels: version: v2
18

Load Balancing Strategies

// SPREADING TRAFFIC INTELLIGENTLY
StrategyHow it WorksWhere it Applies in K8s
Round Robin Each request goes to the next server in rotation kube-proxy default (iptables), gRPC headless services
Least Connections Route to the server with fewest active connections IPVS mode kube-proxy, nginx upstream
IP Hash / Sticky Hash client IP → always route to same server Ingress session affinity, stateful WebSockets
Weighted Send X% traffic to v1, Y% to v2 Istio VirtualService weights, Argo Rollouts
Random Random selection from healthy pods Envoy/Linkerd proxy default
Health-aware Only route to pods passing readiness probes K8s Endpoints controller (automatic)
▸
kube-proxy IPVS vs iptables: The default kube-proxy mode is iptables, which does random selection among Endpoints. At large scale (1000+ services), iptables O(n) rule matching becomes a bottleneck. Enable IPVS mode for O(1) lookup using kernel-level load balancing with more algorithm choices (rr, lc, dh, sh, sed, nq).
19

Network Policies — Zero Trust in K8s

// DENY BY DEFAULT, ALLOW EXPLICITLY

Zero Trust networking in Kubernetes means: no pod is implicitly trusted, every communication is explicitly authorized. Start with a default-deny policy on every namespace, then carve out the minimum required paths.

yaml — Production Zero Trust NetworkPolicy Template
## Step 1: Apply to every namespace — deny everything apiVersion: networking.k8s.io/v1 kind: NetworkPolicy metadata: name: 00-default-deny namespace: production spec: podSelector: {} policyTypes: [Ingress, Egress] --- ## Step 2: Allow DNS (required for name resolution) apiVersion: networking.k8s.io/v1 kind: NetworkPolicy metadata: name: 01-allow-dns namespace: production spec: podSelector: {} policyTypes: [Egress] egress: - to: - namespaceSelector: matchLabels: kubernetes.io/metadata.name: kube-system ports: - protocol: UDP port: 53 - protocol: TCP port: 53 --- ## Step 3: Allow ingress controller → frontend pods apiVersion: networking.k8s.io/v1 kind: NetworkPolicy metadata: name: 02-allow-ingress-to-frontend namespace: production spec: podSelector: matchLabels: tier: frontend policyTypes: [Ingress] ingress: - from: - namespaceSelector: matchLabels: kubernetes.io/metadata.name: ingress-nginx ports: - port: 3000 --- ## Step 4: Frontend → API only apiVersion: networking.k8s.io/v1 kind: NetworkPolicy metadata: name: 03-frontend-to-api namespace: production spec: podSelector: matchLabels: tier: api policyTypes: [Ingress] ingress: - from: - podSelector: matchLabels: tier: frontend ports: - port: 8080 --- ## Step 5: API → database only apiVersion: networking.k8s.io/v1 kind: NetworkPolicy metadata: name: 04-api-to-database namespace: production spec: podSelector: matchLabels: tier: database policyTypes: [Ingress] ingress: - from: - podSelector: matchLabels: tier: api ports: - port: 5432
20

Monitoring & Troubleshooting

// DEBUGGING THE INVISIBLE NETWORK
bash — Network Troubleshooting Toolkit
## ─── SERVICE CONNECTIVITY ─────────────────────────────── # Can a pod reach a service? kubectl exec -it my-pod -- curl -v http://api-svc:8080/health # Test DNS resolution from inside pod kubectl exec -it my-pod -- nslookup api-svc kubectl exec -it my-pod -- cat /etc/resolv.conf # Run a debug pod with network tools (busybox / netshoot) kubectl run netdebug --image=nicolaka/netshoot --rm -it -- bash # Then inside: curl, nslookup, dig, tcpdump, ss, nmap, mtr ## ─── ENDPOINTS & SERVICES ─────────────────────────────── # Check if Service has any Endpoints (are pods selected?) kubectl get endpoints api-svc -n production # Empty ENDPOINTS = pods not matching label selector → check labels # Verify pod labels match Service selector kubectl get pods -l app=api -n production kubectl describe svc api-svc -n production | grep Selector ## ─── INGRESS DEBUGGING ────────────────────────────────── # Check Ingress resource kubectl describe ingress app-ingress -n production # Check nginx controller logs kubectl logs -n ingress-nginx deploy/ingress-nginx-controller --tail=50 # Check if TLS cert is provisioned kubectl describe certificate myapp-tls -n production kubectl get secret myapp-tls-secret -n production ## ─── POD NETWORK ──────────────────────────────────────── # Get pod IP kubectl get pod my-pod -o jsonpath='{.status.podIP}' # Can node ping pod directly? ping $(kubectl get pod my-pod -o jsonpath='{.status.podIP}') # View network interfaces inside pod kubectl exec -it my-pod -- ip addr # Capture traffic on pod (requires netshoot or debug container) kubectl exec -it my-pod -- tcpdump -i eth0 -n port 8080 ## ─── NETWORK POLICY DEBUGGING ─────────────────────────── # List all NetworkPolicies in namespace kubectl get networkpolicies -n production # Check if Cilium is enforcing (Cilium CLI) cilium connectivity test cilium endpoint list

Common Networking Problems & Solutions

SymptomLikely CauseFix
connection refused on serviceNo pods selected by service (empty Endpoints)Check pod labels match service selector
no such host DNS failureCoreDNS not running, or DNS egress blockedCheck CoreDNS pods; add DNS egress NetworkPolicy
Intermittent timeoutNetworkPolicy blocking some pods; one healthy pod, others blockedAudit NetworkPolicies with kubectl describe
Ingress returns 502Backend pods not ready; wrong targetPortCheck pod readinessProbe; verify port mapping
TLS cert not workingcert-manager not provisioning; wrong secret nameCheck cert-manager logs; kubectl describe certificate
gRPC load balancing brokenSingle long-lived HTTP/2 connection hits one podUse headless service + client-side LB or service mesh
External IP pending foreverCloud LB controller not installed / configuredCheck cloud-controller-manager; verify IAM permissions
Pod can't reach internetEgress NetworkPolicy blocking all external trafficAdd explicit egress rule for external IP ranges
21

Network Security

// HARDENING YOUR CLUSTER NETWORK
TLS Everywhere
  • Use cert-manager + Let's Encrypt for automatic TLS on Ingress
  • Enable mTLS between services (via service mesh or manual)
  • Never expose internal services without TLS in production
  • Rotate certificates automatically (cert-manager handles this)
  • Enforce TLS 1.2+ minimum; disable TLS 1.0/1.1
API Server Protection
  • Never expose kube-apiserver (port 6443) to the internet
  • Use a VPN or bastion host for cluster access
  • Enable audit logging on the API server
  • Use RBAC — no wildcards in production ClusterRoles
  • Restrict access with --anonymous-auth=false
Pod Security
  • Set automountServiceAccountToken: false if not needed
  • Use read-only root filesystem where possible
  • Don't run containers as root (runAsNonRoot: true)
  • Drop all Linux capabilities, add back only required ones
  • Use seccomp and AppArmor profiles
Ingress Security
  • Rate limiting on all public-facing Ingress resources
  • WAF (Web Application Firewall) in front of Ingress
  • Use allowPrivilegeEscalation: false on ingress pods
  • Block access to internal metadata endpoints (169.254.169.254)
  • Implement CORS headers — never use wildcard * in production
22

Quick Reference

// COMMANDS YOU'LL USE EVERY DAY
Docker Networking Commands
bash
# List networks docker network ls # Create user-defined bridge docker network create mynet # Run with port mapping docker run -p 8080:80 nginx # Inspect network docker network inspect mynet # Connect to network docker network connect mynet mycontainer # Compose: restart service docker compose restart api
Kubernetes Service Commands
bash
# List all services kubectl get svc -A # Expose a deployment kubectl expose deploy myapp --port 80 --target-port 8080 # Port-forward for local access kubectl port-forward svc/api-svc 8080:80 # Get endpoints kubectl get endpoints api-svc # Describe ingress kubectl describe ingress app-ingress -n prod # Get network policies kubectl get netpol -n prod

K8s DNS Cheat Sheet

DNS Patterns
# Service DNS format: <service-name>.<namespace>.svc.cluster.local # Same namespace shorthand (works due to search domains): api-svc # resolves to api-svc.default.svc.cluster.local # Cross-namespace: api-svc.production # resolves to api-svc.production.svc.cluster.local # StatefulSet pod DNS: kafka-0.kafka-headless.production.svc.cluster.local kafka-1.kafka-headless.production.svc.cluster.local # CoreDNS ClusterIP (always): 10.96.0.10 # Common service CIDRs: 10.96.0.0/12 # K8s service CIDR (default kubeadm) 10.244.0.0/16 # pod CIDR (Flannel default) 10.217.0.0/16 # pod CIDR (Cilium default) 192.168.0.0/16 # pod CIDR (Calico default)
▸
Essential reads: Kubernetes Networking Book (Calico docs) · kubernetes.io/docs/concepts/services-networking/ · Cilium Network Policy Editor (editor.cilium.io) · Cloudflare Learning — How Does DNS Work.