TECH **Docker Compose: Zero-Fluff, Hands-On Guide**

Docker Compose: Zero-Fluff, Hands-On Guide

(Services, Networks, Volumes, Environment)

1. What This Is & Why It Matters

Docker Compose is your local development and small-scale deployment Swiss Army knife. It lets you define and run multi-container Docker applications with a single YAML file (docker-compose.yml). Think of it as a blueprint for your entire app stack—web server, database, cache, and all—so you can spin it up with one command (docker-compose up).

Why this matters in production:
- Reproducibility: No more "works on my machine" bugs. Your docker-compose.yml is the single source of truth for your app’s environment.
- Isolation: Each service runs in its own container, with its own network and storage, just like in Kubernetes (but simpler).
- CI/CD & Testing: Use Compose to run integration tests in a near-production environment before deploying to Kubernetes.
- Legacy App Lift-and-Shift: If you inherit a monolith that "just works" on a VM, Compose lets you containerize it without rewriting everything.

Real-world scenario:
You’re a DevOps engineer at a startup. The dev team just handed you a Python Flask app with a Redis cache and PostgreSQL DB. They run it locally with python app.py, but it crashes in staging because the DB version is wrong. Your mission: Containerize the entire stack with Compose so it runs identically everywhere—local dev, CI, and staging.

2. Core Concepts & Components

1. services

• Definition: The individual containers in your app (e.g., web, db, redis).
• Production insight: Each service should do one thing well (e.g., web handles HTTP, db handles storage). Avoid "fat containers" that run multiple processes.

2. networks

• Definition: Virtual networks that connect your services. By default, Compose creates a single network for all services.
• Production insight: Use custom networks to isolate sensitive services (e.g., db on a private network, web on a public one). In Kubernetes, this maps to NetworkPolicy.

3. volumes

• Definition: Persistent storage for containers. Without volumes, data disappears when the container stops.
• Production insight: Use named volumes for databases (e.g., postgres_data) and bind mounts for local development (e.g., mounting your code into the container). In Kubernetes, this maps to PersistentVolume (PV) and PersistentVolumeClaim (PVC).

4. environment

• Definition: Environment variables passed to containers (e.g., POSTGRES_PASSWORD=secret).
• Production insight: Never hardcode secrets in docker-compose.yml. Use .env files or secrets management (e.g., Docker Secrets, AWS Secrets Manager). In Kubernetes, this maps to ConfigMap and Secret.

5. depends_on

• Definition: Controls the startup order of services (e.g., web depends on db).
• Production insight: depends_on only waits for the container to start, not for the service to be ready (e.g., PostgreSQL might still be initializing). Use health checks (healthcheck) for true readiness.

6. ports

• Definition: Maps host ports to container ports (e.g., "8080:80").
• Production insight: Avoid exposing ports in production. Use a reverse proxy (e.g., Nginx, Traefik) or Kubernetes Ingress.

7. build

• Definition: Builds a Docker image from a Dockerfile for a service.
• Production insight: In production, pre-build images and push them to a registry (e.g., Docker Hub, ECR). Only use build in development.

8. restart

• Definition: Defines the restart policy for a container (e.g., always, on-failure).
• Production insight: Use restart: unless-stopped for critical services. In Kubernetes, this maps to restartPolicy.

3. Step-by-Step Hands-On: Containerize a Flask + Redis + PostgreSQL App

Prerequisites

• Docker installed (Install Docker)
• Docker Compose installed (comes with Docker Desktop; otherwise, install it)
• A Flask app (we’ll use a simple one below)

Step 1: Project Structure

Create this directory structure:

flask-app/
├── app/
│   ├── __init__.py
│   ├── app.py
│   └── requirements.txt
├── docker-compose.yml
└── .env

Step 2: Write the Flask App

app/app.py:

from flask import Flask
import redis
import psycopg2
import os

app = Flask(__name__)

# Redis
redis_host = os.getenv("REDIS_HOST", "redis")
redis_port = os.getenv("REDIS_PORT", 6379)
r = redis.Redis(host=redis_host, port=redis_port, db=0)

# PostgreSQL
pg_host = os.getenv("POSTGRES_HOST", "db")
pg_user = os.getenv("POSTGRES_USER", "postgres")
pg_password = os.getenv("POSTGRES_PASSWORD", "postgres")
pg_db = os.getenv("POSTGRES_DB", "postgres")

def get_db_connection():
    conn = psycopg2.connect(
        host=pg_host,
        user=pg_user,
        password=pg_password,
        database=pg_db
    )
    return conn

@app.route("/")
def hello():
    # Increment counter in Redis
    r.incr("hits")
    hits = r.get("hits").decode("utf-8")

    # Query PostgreSQL
    conn = get_db_connection()
    cur = conn.cursor()
    cur.execute("SELECT version();")
    pg_version = cur.fetchone()[0]
    cur.close()
    conn.close()

    return f"Hello! Redis hits: {hits}. PostgreSQL version: {pg_version}"

if __name__ == "__main__":
    app.run(host="0.0.0.0", port=5000)

app/requirements.txt:

flask==2.0.1
redis==4.1.0
psycopg2-binary==2.9.3

Step 3: Write the docker-compose.yml

version: "3.8"

services:
  web:
    build: .
    ports:
      - "5000:5000"
    environment:
      - REDIS_HOST=redis
      - POSTGRES_HOST=db
      - POSTGRES_USER=${POSTGRES_USER}
      - POSTGRES_PASSWORD=${POSTGRES_PASSWORD}
      - POSTGRES_DB=${POSTGRES_DB}
    depends_on:
      - redis
      - db
    networks:
      - app_network

  redis:
    image: "redis:alpine"
    ports:
      - "6379:6379"
    volumes:
      - redis_data:/data
    networks:
      - app_network

  db:
    image: "postgres:13-alpine"
    environment:
      - POSTGRES_USER=${POSTGRES_USER}
      - POSTGRES_PASSWORD=${POSTGRES_PASSWORD}
      - POSTGRES_DB=${POSTGRES_DB}
    volumes:
      - postgres_data:/var/lib/postgresql/data
    networks:
      - app_network
    healthcheck:
      test: ["CMD-SHELL", "pg_isready -U ${POSTGRES_USER}"]
      interval: 5s
      timeout: 5s
      retries: 5

volumes:
  redis_data:
  postgres_data:

networks:
  app_network:
    driver: bridge

Step 4: Create .env File

POSTGRES_USER=postgres
POSTGRES_PASSWORD=mysecretpassword
POSTGRES_DB=mydb

Step 5: Build and Run

# Build and start the stack
docker-compose up -d

# Check logs
docker-compose logs -f web

# Verify it works
curl http://localhost:5000
# Output: "Hello! Redis hits: 1. PostgreSQL version: PostgreSQL 13.4 on x86_64-pc-linux-musl, compiled by gcc (Alpine 10.3.1_git20210424) 10.3.1 20210424, 64-bit"

# Stop the stack
docker-compose down

Step 6: Verify Data Persistence

# Stop the stack
docker-compose down

# Start it again
docker-compose up -d

# Check the counter (should increment)
curl http://localhost:5000
# Output: "Hello! Redis hits: 2. PostgreSQL version: PostgreSQL 13.4..."

Why this works:
- Volumes (redis_data, postgres_data) persist data between restarts.
- The healthcheck ensures PostgreSQL is ready before web starts.
- Environment variables are loaded from .env.

4. ? Production-Ready Best Practices

Security

• Never commit .env to Git. Add it to .gitignore.
• Use Docker Secrets for sensitive data (e.g., POSTGRES_PASSWORD). In Kubernetes, use Secret.
• Restrict network access: Use custom networks to isolate services (e.g., db should not be exposed to the host).
• Scan images for vulnerabilities: Use docker scan or tools like Trivy.

Cost Optimization

• Use alpine-based images (e.g., postgres:13-alpine) to reduce image size.
• Set resource limits in docker-compose.yml: yaml deploy:
resources:
limits:
cpus: "0.5"
memory: 512M
• Clean up unused containers/volumes:
  bash docker system prune -a --volumes

Reliability & Maintainability

• Use versioned images (e.g., postgres:13-alpine instead of postgres:latest).
• Tag your Compose files (e.g., docker-compose.prod.yml) for different environments.
• Use restart: unless-stopped for critical services.
• Add health checks to all services: yaml healthcheck:
test: ["CMD", "curl", "-f", "http://localhost:5000"]
interval: 30s
timeout: 10s
retries: 3

Observability

• Log to stdout/stderr: Docker captures these by default. Avoid logging to files inside containers.
• Use docker-compose logs for debugging:
  bash docker-compose logs -f --tail=100 web
• Monitor resource usage:
  bash docker stats

5. ⚠️ Common Mistakes & Traps

6. ? Exam/Certification Focus

Typical Question Patterns

1. Port Mapping:
2. Question: "You have a service running on port 80 in a container. How do you expose it on port 8080 on the host?"
3. Answer: "8080:80" in ports.

4. Trap: "80:8080" (host:container, not container:host).

5. Volumes:

6. Question: "How do you persist PostgreSQL data between container restarts?"
7. Answer: Use a named volume (e.g., postgres_data:/var/lib/postgresql/data).

8. Trap: Using a bind mount (e.g., ./data:/var/lib/postgresql/data) for production (slower, less portable).

9. Environment Variables:

10. Question: "How do you pass POSTGRES_PASSWORD=secret to a service?"
11. Answer: Use environment: or .env file.

12. Trap: Hardcoding secrets in docker-compose.yml.

13. Networks:

14. Question: "How do you isolate the db service from the web service?"
15. Answer: Use custom networks (e.g., db_network for db, web_network for web).

16. Trap: Relying on the default network (all services can talk to each other).

17. Health Checks:

18. Question: "How do you ensure web only starts after db is ready?"
19. Answer: Use depends_on + healthcheck.
20. Trap: Using only depends_on (doesn’t wait for readiness).

7. ? Hands-On Challenge

Challenge:
You have a docker-compose.yml with a web service (Flask) and a db service (PostgreSQL). The web service crashes on startup because it tries to connect to db before PostgreSQL is ready. Fix it.

Solution:

services:
  web:
    depends_on:
      db:
        condition: service_healthy
  db:
    healthcheck:
      test: ["CMD-SHELL", "pg_isready -U postgres"]
      interval: 5s
      timeout: 5s
      retries: 5

Why it works:
- depends_on with condition: service_healthy waits for the db health check to pass.
- The healthcheck ensures PostgreSQL is ready before web starts.

8. ? Rapid-Reference Crib Sheet

9. ? Where to Go Next

1. Docker Compose Official Docs – The best reference.
2. Docker Compose File Reference – Full spec for docker-compose.yml.
3. Docker & Kubernetes: The Practical Guide (Udemy) – Hands-on course.
4. 12 Factor App: Config – Best practices for environment variables.