Data Science and Machine Learning 101: Model Deployment and MLOps Containerization and Orchestration Docker Kubernetes Model Packaging

What This Is

Containerization bundles your code, runtime, libraries, and system tools into a portable image that runs the same everywhere—your laptop, a cloud VM, or a CI/CD server. Orchestration (e.g., Kubernetes) then schedules those containers, scales them, and keeps them healthy. For a data‑science team this means a churn‑prediction model can be trained locally, packaged once, and deployed to a production API that serves millions of requests without “it works on my machine” bugs.

Key Terms & Formulas

• Dockerfile – A text script that declares the base OS, Python version, and RUN pip install … steps to build a Docker image.
• Image → Container – An immutable image (layers of filesystem changes) becomes a runnable container when Docker starts a process inside it.
• docker build -t mymodel:1.0 . – Command that reads the Dockerfile in the current directory (.) and tags the resulting image as mymodel:1.0.
• docker run -p 8080:80 mymodel:1.0 – Starts a container, mapping host port 8080 → container port 80 (typical for a FastAPI/Flask model server).
• Kubernetes Pod – The smallest deployable unit; a pod can hold one or more tightly‑coupled containers (e.g., model server + side‑car logger).
• Deployment → ReplicaSet → Pod – A Deployment declares the desired number of pod replicas; Kubernetes creates a ReplicaSet to enforce that count, which in turn creates the pods.
• Horizontal Pod Autoscaler (HPA) – Formula: desiredReplicas = ceil(currentCPUUtilization / targetCPUUtilization * currentReplicas). Adjusts pod count based on metrics (CPU, custom latency).
• kubectl apply -f deployment.yaml – Imperative command that makes the cluster state match the YAML manifest (declarative infra).
• Model Packaging (MLflow / BentoML) – Stores model artifacts, conda/pip env, and a Dockerfile template; mlflow models build-docker -m run_id -n mymodel:latest.
• ENTRYPOINT ["python", "serve.py"] – Docker instruction that defines the default command when the container starts (often a model‑serving script).
• Resource Limits – resources: limits: cpu: "2" tells Kubernetes not to schedule the pod on a node that can’t guarantee 2 CPU cores.
• Rolling Update – Deployment strategy that replaces old pods with new ones gradually, keeping at least maxUnavailable pods alive (zero‑downtime).

Step‑by‑Step / Process Flow

1. Develop & Test Locally
   python
# train.py
import pandas as pd, joblib, sklearn
df = pd.read_csv('train.csv')
X, y = df.drop('target', axis=1), df['target']
model = sklearn.ensemble.RandomForestClassifier(n_estimators=200)
model.fit(X, y)
joblib.dump(model, 'model.pkl')
2. Create a Minimal Dockerfile
   Dockerfile
FROM python:3.11-slim
WORKDIR /app
COPY requirements.txt .
RUN pip install -r requirements.txt
COPY model.pkl serve.py .
ENTRYPOINT ["python", "serve.py"]
3. Build & Verify the Image
   bash
docker build -t churn-model:0.1 .
docker run -p 8000:8000 churn-model:0.1 # hit http://localhost:8000/predict
4. Push to a Registry (Docker Hub, ECR, GCR)
   bash
docker tag churn-model:0.1 myrepo/churn-model:0.1
docker push myrepo/churn-model:0.1
5. Write a Kubernetes Deployment Manifest
   yaml
apiVersion: apps/v1
kind: Deployment
metadata:
name: churn-svc
spec:
replicas: 2
selector:
matchLabels: {app: churn}
template:
metadata:
labels: {app: churn}
spec:
containers:
- name: churn
image: myrepo/churn-model:0.1
ports: [{containerPort: 8000}]
resources:
limits: {cpu: "1", memory: "1Gi"}
6. Deploy & Autoscale
   bash
kubectl apply -f deployment.yaml
kubectl autoscale deployment churn-svc --cpu-percent=70 --min=2 --max=10

Common Mistakes

Data Science Interview / Practical Insights

1. “Why containerize a model instead of just sharing a .pkl file?” – Expect you to discuss reproducibility, dependency isolation, and CI/CD pipelines.
2. “Explain the difference between a Docker image and a container.” – Image = static snapshot; container = running instance with its own PID, network, and storage layers.
3. “When would you choose a RollingUpdate vs a Recreate strategy?” – RollingUpdate for zero‑downtime services; Recreate when stateful pods cannot run concurrently (e.g., a database).
4. “How do you expose a model as a REST API inside a container?” – Mention FastAPI/Flask, uvicorn --host 0.0.0.0 --port 8000 serve:app, and the need for EXPOSE 8000 in Dockerfile.

Quick Check Questions

1. Scenario: Your model server crashes after a few requests, but the Docker image runs fine locally.
   Answer: Check Kubernetes livenessProbe and resource limits; the pod may be OOM‑killed because the memory limit is too low.

2. Scenario: You need to roll out a new model version without downtime.
   Answer: Use a RollingUpdate Deployment with maxSurge: 1 and maxUnavailable: 0 so a new pod starts before the old one stops.

3. Scenario: A teammate asks why the model predictions differ between dev and prod.
   Answer: Likely a dependency version mismatch; containers guarantee identical libraries, eliminating this source of drift.

Last‑Minute Cram Sheet (10 one‑liners)

1. Docker image = layered filesystem; each RUN creates a new immutable layer.
2. docker exec -it <container> bash lets you debug inside a running container.
3. ENTRYPOINT vs CMD: ENTRYPOINT is the fixed executable; CMD supplies default args that can be overridden.
4. kubectl rollout status deployment/<name> shows progress of a rolling update.
5. ⚠️ Forgetting EXPOSE does NOT publish ports; you still need -p host:container.
6. resources.limits are hard caps; resources.requests are the scheduler’s guarantee.
7. MLflow’s mlflow models build-docker auto‑generates a Dockerfile that pins the exact Python env.
8. HPA scales on CPU by default; add a custom metric (e.g., request latency) for better autoscaling of model APIs.
9. docker prune -a removes dangling images and frees disk space—useful after many builds.
10. Kubernetes pod IPs are ephemeral; always reach services via a Service (ClusterIP, LoadBalancer).