TECH **Docker & Kubernetes Volumes & PersistentVolumeClaims: Zero-Fluff Study Guide**

Docker & Kubernetes Volumes & PersistentVolumeClaims: Zero-Fluff Study Guide

For engineers who need to deploy stateful apps in production—yesterday.

1. What This Is & Why It Matters

You’re deploying a database, a logging agent, or a user-uploaded file service in Kubernetes. If you don’t configure storage correctly: - Your data disappears when the pod restarts (because containers are ephemeral).
- Your app crashes if it can’t write to disk (e.g., PostgreSQL, Redis, or a WordPress uploads folder).
- You violate compliance (e.g., GDPR, HIPAA) by storing sensitive data in the wrong place.

Real-world scenario:
You inherit a Kubernetes cluster where a legacy app stores logs in /var/log/app inside the container. When the pod crashes, the logs vanish. Your boss asks, "Where are yesterday’s logs?" You need to persist data outside the pod—this guide shows you how.

Superpower you gain:
- Decouple compute from storage (pods come and go; data stays).
- Share data between pods (e.g., a shared config file or a database volume).
- Control performance, cost, and durability (SSD vs. HDD, local vs. cloud storage).

2. Core Concepts & Components

1. Volume (Kubernetes)

• Definition: A directory accessible to containers in a pod, backed by storage (local, network, or cloud).
• Production insight: Volumes are pod-scoped—if the pod dies, the volume dies unless it’s backed by persistent storage (e.g., PersistentVolume).

2. emptyDir

• Definition: A temporary, empty directory created when a pod starts and deleted when the pod terminates.
• Use case: Scratch space for caching, temporary files, or sharing data between containers in the same pod.
• Production insight: Not for production data—use it for ephemeral workloads (e.g., a sidecar container processing logs). If the node crashes, the data is lost.

3. hostPath

• Definition: Mounts a file or directory from the host node’s filesystem into the pod.
• Use case: Accessing node-level files (e.g., Docker socket, /dev devices, or logs).
• Production insight: Security risk—hostPath gives pods access to the host’s filesystem. Use readOnly: true and restrict to trusted pods (e.g., monitoring agents).

4. PersistentVolume (PV)

• Definition: A cluster-wide storage resource provisioned by an admin (e.g., an EBS volume, NFS share, or local SSD).
• Production insight: PVs are independent of pods—they persist even if the pod using them dies. Think of it like a network-attached hard drive for your cluster.

5. PersistentVolumeClaim (PVC)

• Definition: A request for storage by a user (e.g., "I need 10Gi of SSD storage"). Kubernetes binds the PVC to a matching PV.
• Production insight: PVCs decouple storage requests from provisioning—developers don’t need to know where the storage comes from (EBS, NFS, etc.).

6. StorageClass

• Definition: Defines dynamic provisioning of PVs (e.g., "Create a new EBS volume when a PVC requests fast-ssd storage").
• Production insight: Critical for cost control—set default StorageClasses to avoid expensive storage (e.g., gp3 instead of io1).

7. Access Modes

• ReadWriteOnce (RWO): One pod can read/write (e.g., databases).
• ReadOnlyMany (ROX): Multiple pods can read (e.g., config files).
• ReadWriteMany (RWX): Multiple pods can read/write (e.g., shared logs, NFS).
• Production insight: Not all storage backends support all modes (e.g., EBS only supports RWO; NFS supports RWX).

8. Reclaim Policy

• Retain: PV is kept after PVC is deleted (manual cleanup required).
• Delete: PV is deleted when PVC is deleted (default for dynamically provisioned PVs).
• Recycle: Deprecated (used to scrub data and make PV available again).
• Production insight: Use Retain for critical data—otherwise, you might accidentally delete production databases.

3. Step-by-Step Hands-On: Deploy a Stateful App with PVCs

Prerequisites

• A running Kubernetes cluster (Minikube, Kind, or cloud-based).
• kubectl configured to talk to your cluster.
• A StorageClass (most clusters have a default one, e.g., standard in GKE or gp2 in EKS).

Task: Deploy PostgreSQL with Persistent Storage

We’ll: 1. Create a PVC requesting 5Gi of storage.
2. Deploy PostgreSQL with the PVC mounted at /var/lib/postgresql/data.
3. Verify data persists after pod deletion.

Step 1: Create a PersistentVolumeClaim (PVC)

# pvc.yaml
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
  name: postgres-pvc
spec:
  accessModes:
    - ReadWriteOnce
  resources:
    requests:
      storage: 5Gi
  storageClassName: standard  # Use your cluster's default StorageClass

Apply it:

kubectl apply -f pvc.yaml

Verify:

kubectl get pvc

Expected output:

NAME           STATUS   VOLUME                                     CAPACITY   ACCESS MODES   STORAGECLASS   AGE
postgres-pvc   Bound    pvc-12345678-1234-1234-1234-1234567890ab   5Gi        RWO            standard       5s

Step 2: Deploy PostgreSQL with the PVC

# postgres-deployment.yaml
apiVersion: apps/v1
kind: Deployment
metadata:
  name: postgres
spec:
  replicas: 1
  selector:
    matchLabels:
      app: postgres
  template:
    metadata:
      labels:
        app: postgres
    spec:
      containers:
      - name: postgres
        image: postgres:13
        env:
        - name: POSTGRES_PASSWORD
          value: "mysecretpassword"
        ports:
        - containerPort: 5432
        volumeMounts:
        - name: postgres-storage
          mountPath: /var/lib/postgresql/data
      volumes:
      - name: postgres-storage
        persistentVolumeClaim:
          claimName: postgres-pvc

Apply it:

kubectl apply -f postgres-deployment.yaml

Verify:

kubectl get pods

Expected output:

NAME                        READY   STATUS    RESTARTS   AGE
postgres-5f7d8c6b8c-abc12   1/1     Running   0          10s

Step 3: Test Data Persistence

1. Create a test table:
   bash
kubectl exec -it postgres-5f7d8c6b8c-abc12 -- psql -U postgres
   Inside the PostgreSQL shell:
   sql
CREATE TABLE test (id serial PRIMARY KEY, name text);
INSERT INTO test (name) VALUES ('persistent-data');
\q

2. Delete the pod (simulate a crash):
   bash
kubectl delete pod postgres-5f7d8c6b8c-abc12

3. Verify the data is still there:
   bash
kubectl get pods # Wait for the new pod to start
kubectl exec -it postgres-5f7d8c6b8c-xyz78 -- psql -U postgres -c "SELECT * FROM test;"
   Expected output:
   id | name
----+----------------
1 | persistent-data

Step 4: Clean Up

kubectl delete deployment postgres
kubectl delete pvc postgres-pvc

4. ? Production-Ready Best Practices

Security

• Never use hostPath for untrusted workloads—it gives pods access to the host’s filesystem.
• Set readOnly: true for volumes that don’t need writes (e.g., config files).
• Use fsGroup to ensure files are owned by the correct user (e.g., PostgreSQL runs as postgres user): yaml securityContext:
fsGroup: 999 # PostgreSQL's default user ID

Cost Optimization

• Use the right StorageClass (e.g., gp3 instead of io1 for EBS).
• Set resource limits on PVCs to avoid over-provisioning: yaml resources:
requests:
storage: 10Gi
limits:
storage: 20Gi # Prevents runaway growth
• Use ReadWriteMany (RWX) for shared storage (e.g., NFS) to avoid duplicating data.

Reliability & Maintainability

• Label your PVCs for easy filtering: yaml metadata:
labels:
app: postgres
env: prod
• Use volumeMode: Filesystem (default) or Block for raw block storage (e.g., databases).
• Set reclaimPolicy: Retain for critical data to prevent accidental deletion.

Observability

• Monitor PVC usage with Prometheus + Grafana: bash kubectl top pvc # Requires metrics-server
• Set alerts for PVCs nearing capacity (e.g., 80% full).
• Log volume mount errors (e.g., "Permission denied" when fsGroup is misconfigured).

5. ⚠️ Common Mistakes & Traps

6. ? Exam/Certification Focus

Typical Question Patterns

1. Scenario: "Your app needs shared storage for logs. Which access mode should you use?"
2. Answer: ReadWriteMany (RWX).

3. Trap: ReadWriteOnce (RWO) won’t work for multiple pods.

4. Scenario: "You delete a PVC, but the PV is still there. Why?"

5. Answer: The PV’s reclaimPolicy is Retain.

6. Trap: Delete would have removed the PV.

7. Scenario: "Your PVC is stuck in Pending. What’s the most likely cause?"

8. Answer: No matching PV or StorageClass.
9. Trap: Assuming the PVC will auto-create a PV (it won’t unless dynamic provisioning is set up).

Key Distinctions

7. ? Hands-On Challenge

Challenge:
Deploy a Redis pod with a PVC. Write a key-value pair (SET mykey "hello"), delete the pod, and verify the data persists.

Solution:
1. Create a PVC:
yaml
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
name: redis-pvc
spec:
accessModes:
- ReadWriteOnce
resources:
requests:
storage: 1Gi 2. Deploy Redis with the PVC:
yaml
apiVersion: apps/v1
kind: Deployment
metadata:
name: redis
spec:
replicas: 1
selector:
matchLabels:
app: redis
template:
metadata:
labels:
app: redis
spec:
containers:
- name: redis
image: redis
volumeMounts:
- name: redis-storage
mountPath: /data
volumes:
- name: redis-storage
persistentVolumeClaim:
claimName: redis-pvc 3. Test persistence:
bash
kubectl exec -it redis-<pod-id> -- redis-cli SET mykey "hello"
kubectl delete pod redis-<pod-id>
kubectl exec -it redis-<new-pod-id> -- redis-cli GET mykey # Should return "hello"

Why it works:
Redis stores data in /data by default. The PVC ensures the data persists across pod restarts.

8. ? Rapid-Reference Crib Sheet

9. ? Where to Go Next

1. Kubernetes Docs: Volumes – Official reference.
2. Kubernetes Docs: Persistent Volumes – Deep dive into PVs and PVCs.
3. StorageClass Deep Dive – Learn how to configure dynamic provisioning.
4. Kubernetes the Hard Way: Persistent Volumes – Hands-on lab for setting up PVs manually.