TECH **Kubernetes Network Policies: Zero-Fluff, Hands-On Guide**

Kubernetes Network Policies: Zero-Fluff, Hands-On Guide

Pod-to-Pod Traffic Control for Engineers Who Ship

1. What This Is & Why It Matters

Network Policies are Kubernetes’ firewall rules for pods. They define which pods can talk to which other pods (or external services) and on which ports. Without them, your cluster is a free-for-all: any pod can reach any other pod, including sensitive ones (databases, internal APIs, admin dashboards).

Why This Matters in Production

• Security: A compromised pod (e.g., a vulnerable frontend) can scan and attack internal services. Network Policies act as a second line of defense after pod security.
• Compliance: PCI-DSS, HIPAA, and SOC2 require micro-segmentation. Network Policies prove you’ve isolated workloads.
• Stability: A misconfigured pod (e.g., a rogue cron job) can DDoS your database. Policies limit blast radius.
• Cost: Unrestricted pod communication leads to noisy neighbors (e.g., a dev pod flooding logs to a production service).

Real-World Scenario

You’re a DevOps engineer at a fintech startup. Your payment service (pod payments-v1) talks to: - A PostgreSQL database (pod pg-primary) - A Redis cache (pod redis-cache) - An internal fraud API (pod fraud-detection)

Problem: A junior dev deploys a debug pod (debug-shell) with kubectl exec access. Without Network Policies, this pod can: - Query the database directly (bypassing the payment service).
- Flood Redis with garbage requests.
- Call the fraud API in a loop, triggering false positives.

Solution: Use Network Policies to enforce: - Only payments-v1 can talk to pg-primary and redis-cache.
- Only payments-v1 and fraud-detection can talk to each other.
- No other pods can reach these services.

2. Core Concepts & Components

1. NetworkPolicy (Resource)

• Definition: A Kubernetes manifest (YAML) that defines ingress/egress rules for pods.
• Production Insight: Network Policies are additive. If no policy exists, all traffic is allowed. If any policy matches a pod, only the traffic explicitly allowed is permitted.

2. podSelector

• Definition: Labels used to select pods to which the policy applies.
• Production Insight: Use specific labels (e.g., app: payments) instead of broad ones (e.g., tier: backend). Overly permissive selectors defeat the purpose.

3. policyTypes

• Definition: Specifies whether the policy applies to Ingress, Egress, or both.
• Production Insight: Always set policyTypes: ["Ingress", "Egress"] unless you have a good reason not to. Default is Ingress only.

4. ingress Rules

• Definition: Defines incoming traffic allowed to the selected pods.
• Production Insight: Use from to restrict sources (pods, namespaces, IP blocks). Omitting from allows traffic from anywhere.

5. egress Rules

• Definition: Defines outgoing traffic allowed from the selected pods.
• Production Insight: Critical for restricting pod access to external services (e.g., only allow payments-v1 to talk to api.stripe.com).

6. namespaceSelector

• Definition: Restricts traffic to/from pods in specific namespaces.
• Production Insight: Useful for multi-tenant clusters (e.g., namespaceSelector: { matchLabels: { env: prod } }).

7. ipBlock

• Definition: Restricts traffic to/from specific CIDR ranges (e.g., 10.0.0.0/24).
• Production Insight: Useful for allowing traffic from on-premises networks or cloud provider services (e.g., AWS VPC endpoints).

8. CNI Plugin Support

• Definition: Network Policies require a CNI plugin that supports them (e.g., Calico, Cilium, WeaveNet). Flannel does not support Network Policies.
• Production Insight: If your cluster uses Flannel, you’ll need to migrate to Calico or Cilium to use Network Policies.

9. Default Deny

• Definition: A policy that blocks all traffic unless explicitly allowed.
• Production Insight: Always start with a default-deny policy for the entire namespace, then add allow rules. This is the only way to ensure no accidental exposure.

3. Step-by-Step Hands-On: Enforcing Pod Isolation

Prerequisites

1. A Kubernetes cluster with a NetworkPolicy-compatible CNI (e.g., Calico, Cilium).
2. Test with: kubectl get pods -n kube-system | grep -E 'calico|cilium'
3. kubectl installed and configured.
4. A test namespace:
   bash
kubectl create namespace network-policy-demo
kubectl config set-context --current --namespace=network-policy-demo

Step 1: Deploy Test Workloads

We’ll simulate the fintech scenario: - payments-v1: The payment service.
- pg-primary: PostgreSQL database.
- redis-cache: Redis cache.
- fraud-detection: Internal fraud API.
- debug-shell: A rogue pod (simulating a compromised pod).

Deploy the pods and services:

# Payments service
kubectl apply -f - <<EOF
apiVersion: v1
kind: Pod
metadata:
  name: payments-v1
  labels:
    app: payments
    tier: backend
spec:
  containers:
  - name: nginx
    image: nginx
    ports:
    - containerPort: 80
---
apiVersion: v1
kind: Service
metadata:
  name: payments
spec:
  selector:
    app: payments
  ports:
    - protocol: TCP
      port: 80
      targetPort: 80
EOF

# PostgreSQL
kubectl apply -f - <<EOF
apiVersion: v1
kind: Pod
metadata:
  name: pg-primary
  labels:
    app: postgres
    tier: db
spec:
  containers:
  - name: postgres
    image: postgres:13
    env:
    - name: POSTGRES_PASSWORD
      value: "password"
    ports:
    - containerPort: 5432
---
apiVersion: v1
kind: Service
metadata:
  name: postgres
spec:
  selector:
    app: postgres
  ports:
    - protocol: TCP
      port: 5432
      targetPort: 5432
EOF

# Redis
kubectl apply -f - <<EOF
apiVersion: v1
kind: Pod
metadata:
  name: redis-cache
  labels:
    app: redis
    tier: cache
spec:
  containers:
  - name: redis
    image: redis
    ports:
    - containerPort: 6379
---
apiVersion: v1
kind: Service
metadata:
  name: redis
spec:
  selector:
    app: redis
  ports:
    - protocol: TCP
      port: 6379
      targetPort: 6379
EOF

# Fraud detection
kubectl apply -f - <<EOF
apiVersion: v1
kind: Pod
metadata:
  name: fraud-detection
  labels:
    app: fraud
    tier: backend
spec:
  containers:
  - name: nginx
    image: nginx
    ports:
    - containerPort: 80
---
apiVersion: v1
kind: Service
metadata:
  name: fraud
spec:
  selector:
    app: fraud
  ports:
    - protocol: TCP
      port: 80
      targetPort: 80
EOF

# Debug shell (rogue pod)
kubectl apply -f - <<EOF
apiVersion: v1
kind: Pod
metadata:
  name: debug-shell
  labels:
    app: debug
    tier: dev
spec:
  containers:
  - name: alpine
    image: alpine
    command: ["sh", "-c", "sleep infinity"]
EOF

Step 2: Verify Unrestricted Traffic

Before applying policies, confirm all pods can talk to each other:

# From debug-shell, try to reach payments, postgres, and redis
kubectl exec debug-shell -- sh -c "nc -zv payments 80 && nc -zv postgres 5432 && nc -zv redis 6379"

Expected Output:

payments (10.96.123.45:80) open
postgres (10.96.123.46:5432) open
redis (10.96.123.47:6379) open

All traffic is allowed by default.

Step 3: Apply Default-Deny Policy

Block all traffic in the namespace:

kubectl apply -f - <<EOF
apiVersion: networking.k8s.io/v1
kind: NetworkPolicy
metadata:
  name: default-deny
spec:
  podSelector: {}
  policyTypes:
  - Ingress
  - Egress
EOF

What this does: - podSelector: {}: Applies to all pods in the namespace.
- policyTypes: ["Ingress", "Egress"]: Blocks both incoming and outgoing traffic.

Verify:

kubectl exec debug-shell -- sh -c "nc -zv payments 80"

Expected Output:

nc: payments (10.96.123.45:80): Operation timed out

Traffic is now blocked.

Step 4: Allow Payments → PostgreSQL & Redis

kubectl apply -f - <<EOF
apiVersion: networking.k8s.io/v1
kind: NetworkPolicy
metadata:
  name: allow-payments-to-db
spec:
  podSelector:
    matchLabels:
      app: payments
  policyTypes:
  - Egress
  egress:
  - to:
    - podSelector:
        matchLabels:
          app: postgres
    ports:
    - protocol: TCP
      port: 5432
  - to:
    - podSelector:
        matchLabels:
          app: redis
    ports:
    - protocol: TCP
      port: 6379
EOF

What this does: - Allows payments-v1 to talk to pg-primary (port 5432) and redis-cache (port 6379).
- No other pods can talk to these services.

Verify:

# From payments-v1, test connectivity to postgres and redis
kubectl exec payments-v1 -- sh -c "nc -zv postgres 5432 && nc -zv redis 6379"

Expected Output:

postgres (10.96.123.46:5432) open
redis (10.96.123.47:6379) open

# From debug-shell, try to reach postgres (should fail)
kubectl exec debug-shell -- sh -c "nc -zv postgres 5432"

Expected Output:

nc: postgres (10.96.123.46:5432): Operation timed out

Step 5: Allow Payments ↔ Fraud Detection

kubectl apply -f - <<EOF
apiVersion: networking.k8s.io/v1
kind: NetworkPolicy
metadata:
  name: allow-payments-fraud
spec:
  podSelector:
    matchLabels:
      app: payments
  policyTypes:
  - Ingress
  - Egress
  ingress:
  - from:
    - podSelector:
        matchLabels:
          app: fraud
    ports:
    - protocol: TCP
      port: 80
  egress:
  - to:
    - podSelector:
        matchLabels:
          app: fraud
    ports:
    - protocol: TCP
      port: 80
EOF

What this does: - Allows fraud-detection to call payments-v1 (ingress).
- Allows payments-v1 to call fraud-detection (egress).

Verify:

# From fraud-detection, test connectivity to payments
kubectl exec fraud-detection -- sh -c "nc -zv payments 80"

Expected Output:

payments (10.96.123.45:80) open

# From debug-shell, try to reach payments (should fail)
kubectl exec debug-shell -- sh -c "nc -zv payments 80"

Expected Output:

nc: payments (10.96.123.45:80): Operation timed out

Step 6: Allow Egress to External Services (e.g., Stripe API)

kubectl apply -f - <<EOF
apiVersion: networking.k8s.io/v1
kind: NetworkPolicy
metadata:
  name: allow-payments-to-stripe
spec:
  podSelector:
    matchLabels:
      app: payments
  policyTypes:
  - Egress
  egress:
  - to:
    - ipBlock:
        cidr: 103.14.0.0/24  # Stripe's IP range (example)
    ports:
    - protocol: TCP
      port: 443
EOF

What this does: - Allows payments-v1 to call api.stripe.com (port 443).
- Replace 103.14.0.0/24 with the actual CIDR of your external service.

Verify:

kubectl exec payments-v1 -- sh -c "nc -zv api.stripe.com 443"

Expected Output:

api.stripe.com (103.14.0.1:443) open

4. ? Production-Ready Best Practices

Security

• Start with default-deny: Always apply a default-deny policy to namespaces before adding allow rules.
• Use specific labels: Avoid broad selectors like tier: backend. Use app: payments instead.
• Restrict egress: Block all egress by default, then allow only necessary external services (e.g., payment gateways, logging services).
• Namespace isolation: Use namespaceSelector to restrict cross-namespace traffic (e.g., only allow prod namespace to talk to monitoring).
• Audit policies: Use tools like kube-score to validate Network Policies.

Cost Optimization

• Avoid overly permissive policies: Every rule adds overhead to the CNI plugin. Keep policies as simple as possible.
• Use ipBlock for external services: Instead of allowing all egress, restrict to specific CIDRs (e.g., AWS S3 endpoints).

Reliability & Maintainability

• Name policies clearly: Use names like allow-payments-to-postgres instead of policy-1.
• Document policies: Add comments in YAML explaining why a rule exists (e.g., # Allows payments to query fraud API for risk checks).
• Test policies in staging: Use tools like k8s-netpol to simulate policies before applying to production.

Observability

• Monitor policy violations: Use CNI plugin logs (e.g., Calico’s felix logs) to detect blocked traffic.
• Alert on unexpected traffic: Set up alerts for traffic that matches default-deny (indicates a missing allow rule).
• Use kubectl describe networkpolicy: Debug policies with: bash kubectl describe networkpolicy allow-payments-to-postgres

5. ⚠️ Common Mistakes & Traps

6. ? Exam/Certification Focus

Typical Question Patterns

1. Default behavior:
2. Question: "What happens if no NetworkPolicy exists in a namespace?"
3. Answer: All traffic is allowed (default-allow).

4. Trap: "All traffic is blocked" (wrong).

5. Policy types:

6. Question: "Which policyTypes must be set to block egress traffic?"
7. Answer: policyTypes: ["Egress"].

8. Trap: Forgetting to include Egress in policyTypes.

9. Selector specificity:

10. Question: "Which podSelector is more secure: tier: backend or app: payments?"
11. Answer: app: payments (more specific).

12. Trap: Using broad selectors like tier: backend.

13. CNI plugin support:

14. Question: "Which CNI plugin does not support Network Policies?"
15. Answer: Flannel.

16. Trap: Assuming all CNI plugins support Network Policies.

17. Scenario-based:

18. Question: "You need to allow traffic from frontend pods to backend pods on port 8080. Which policy do you apply?"
19. Answer:
    ```yaml
    apiVersion: networking.k8s.io/v1
    kind: NetworkPolicy
    metadata:
    name: allow-frontend-to-backend
    spec:
    podSelector:
    matchLabels:
    app: backend
    ingress:
    • from:
      • podSelector:
        matchLabels:
        app: frontend ports:
      • protocol: TCP
        port: 8080 ```

Key ⚠️ Trap Distinctions

• Default-deny vs. default-allow:
• Default-deny: Explicitly block all traffic, then allow specific traffic.
• Default-allow: All traffic is allowed unless blocked (less secure).
• Ingress vs. Egress:
• Ingress: Traffic into the pod.
• Egress: Traffic out of the pod.
• podSelector vs. namespaceSelector:
• podSelector: Matches pods in the same namespace.
• namespaceSelector: Matches pods in other namespaces.

7. ? Hands-On Challenge

Challenge: You have a monitoring namespace with a prometheus pod (labels: app: prometheus). Write a NetworkPolicy that: 1. Allows prometheus