Data Science and Machine Learning 101: Deep Learning and NLP Convolutional Neural Networks CNN for Image Data Conv Layers Pooling Transfer Learning

Convolutional Neural Networks (CNN) for Image Data (Conv Layers, Pooling, Transfer Learning)

What This Is

A Convolutional Neural Network (CNN) is a deep‑learning architecture that automatically learns spatial hierarchies of features from raw pixel grids. In a data‑science workflow it replaces manual feature engineering for images, letting you go from a folder of JPEGs to a production‑ready classifier in a few lines of code. Real‑world example: a hospital uses a CNN to flag suspicious lung nodules on chest X‑rays, reducing radiologist workload and catching disease earlier.

Key Terms & Formulas

• Convolution (∗) – output[i, j, k] = Σ_{c=1}^{C_in} Σ_{u=0}^{K_h-1} Σ_{v=0}^{K_w-1} X[i+u, j+v, c]·W[u, v, c, k] + b_k
  X: input tensor, W: filter bank, K_h/K_w: kernel height/width, C_in: input channels, b: bias. Slides a small filter over the image to produce a feature map.

• Stride (s) – Number of pixels the filter jumps each step. Larger s shrinks the spatial dimensions faster (H_out = ⌊(H_in‑K_h)/s⌋ + 1).

• Padding (p) – Zero‑border added to keep output size. Same padding keeps H_out = ⌈H_in / s⌉.

• ReLU Activation – a = max(0, z). Introduces non‑linearity, speeds up convergence, and mitigates vanishing gradients.

• Pooling – Down‑sampling operation (usually MaxPool): y[i, j, k] = max_{u,v} x[i·s+u, j·s+v, k]. Reduces resolution, adds translation invariance, and cuts memory.

• Batch Normalization – ŷ = γ·(x‑μ)/σ + β. Normalizes each mini‑batch; stabilizes training and allows higher learning rates.

• Transfer Learning – Reuse a pretrained network (e.g., ImageNet‑trained ResNet‑50) as a fixed feature extractor or fine‑tune its top layers. Saves data and compute.

• Cross‑Entropy Loss (CE) – L = - Σ_{c} y_c log(p_c). y: one‑hot true label, p: softmax probability. Drives classification learning.

• Softmax – p_c = exp(z_c) / Σ_{k} exp(z_k). Converts logits to a probability distribution over classes.

• Learning Rate Scheduler – α_t = α_0 / (1 + decay·t) or cosine annealing. Adjusts step size to avoid overshooting late in training.

• Gradient Descent (SGD) – θ ← θ - α·∇_θ L(θ). Updates all trainable parameters θ using the gradient of the loss.

• Fine‑tuning vs. Feature Extraction – Fine‑tuning: unfreeze some deeper layers and continue training; Feature extraction: freeze the whole backbone and only train a new classifier head.

Step‑by‑Step / Process Flow

1. Load & Inspect Images
   python
import pathlib, torchvision
data_dir = pathlib.Path('data/chest_xray')
dataset = torchvision.datasets.ImageFolder(data_dir,
transform=torchvision.transforms.ToTensor())
print(len(dataset), dataset.classes)

2. Pre‑process & Augment – Resize, normalize to ImageNet stats, add random flips/rotations.
   python
transform = torchvision.transforms.Compose([
torchvision.transforms.Resize((224,224)),
torchvision.transforms.RandomHorizontalFlip(),
torchvision.transforms.ToTensor(),
torchvision.transforms.Normalize(mean=[0.485,0.456,0.406],
std=[0.229,0.224,0.225])
])

3. Split Train/Val/Test (e.g., 70/15/15) using torch.utils.data.random_split.

4. Build Model with Transfer Learning
   python
import torch.nn as nn, torchvision.models as models
backbone = models.resnet50(pretrained=True)
for p in backbone.parameters(): # freeze backbone
p.requires_grad = False
backbone.fc = nn.Linear(backbone.fc.in_features, 2) # new head
model = backbone.to(device)

5. Train – Use SGD + momentum, cross‑entropy, and a learning‑rate scheduler.
   python
optimizer = torch.optim.SGD(model.parameters(), lr=0.01, momentum=0.9)
scheduler = torch.optim.lr_scheduler.CosineAnnealingLR(optimizer, T_max=10)
for epoch in range(num_epochs):
train_one_epoch(...)
validate(...)
scheduler.step()

6. Evaluate & Tune – Compute accuracy, ROC‑AUC, and confusion matrix. If over‑fitting, unfreeze deeper layers or add dropout (nn.Dropout(0.5)).

7. Export & Deploy – TorchScript or ONNX for serving; wrap in a Flask API or TensorFlow Serving endpoint.

Common Mistakes

• Mistake: Feeding raw pixel values (0‑255) directly into the network.
  Correction: Normalize to the same mean/std the pretrained model expects; otherwise gradients explode or vanish.

• Mistake: Using a huge kernel (e.g., 11×11) on the first layer of a modern CNN.
  Correction: Stick to 3×3 or 5×5 kernels; they capture local patterns efficiently and keep parameters low.

• Mistake: Freezing all layers and only training a tiny head on a small medical dataset.
  Correction: Fine‑tune at least the last two blocks; the domain shift (natural → medical images) often requires adaptation.

• Mistake: Ignoring class imbalance and optimizing only for overall accuracy.
  Correction: Use weighted cross‑entropy or focal loss, and monitor per‑class recall/precision.

• Mistake: Forgetting to set model.eval() during inference, leaving BatchNorm and Dropout in training mode.
  Correction: Always switch to evaluation mode; otherwise predictions become noisy and non‑deterministic.

Data Science Interview / Practical Insights

1. “Explain why a CNN is better than a fully‑connected MLP for images.” – Talk about parameter sharing, local receptive fields, and translation invariance.

2. “When would you prefer a pretrained model vs. training from scratch?” – Mention dataset size (< 10k images), compute budget, and similarity to the pretraining domain.

3. “What is the effect of stride > 1 on feature map size and receptive field?” – Larger stride reduces spatial resolution faster but also enlarges the effective receptive field per layer.

4. “How does Global Average Pooling differ from Flatten + Dense, and why is it often used in modern CNNs?” – It reduces parameters, forces the network to learn spatially‑global descriptors, and mitigates over‑fitting.

Quick Check Questions

1. Scenario: Your validation loss keeps rising while training loss keeps dropping.
   Answer: The model is over‑fitting; add regularization (dropout, weight decay) or unfreeze fewer layers.

2. Scenario: You have 5 % positive cases in a chest‑X‑ray dataset. Accuracy is 96 % but recall is 10 %.
   Answer: Accuracy is misleading; switch to a loss that handles imbalance (e.g., focal loss) and monitor ROC‑AUC or PR‑AUC.

3. Scenario: You need to deploy a model on a mobile device with < 2 MB size.
   Answer: Use model compression (pruning, quantization) or a lightweight architecture like MobileNet‑V2.

Last‑Minute Cram Sheet (10 one‑liners)

1. Convolution → Parameter sharing + sparse connectivity = far fewer weights than a dense layer.
2. ReLU ≈ max(0, x); avoids vanishing gradients unlike sigmoid/tanh.
3. MaxPool with stride = kernel size halves height & width each time.
4. BatchNorm normalizes per‑batch; during inference it uses running mean/var.
5. Transfer Learning: freeze early layers, fine‑tune later ones for domain shift.
6. Cross‑Entropy = –Σ y·log(p); the gradient w.r.t. logits is (p – y).
7. Global AvgPool replaces Flatten → Dense, cutting parameters by ~90 %.
8. Weight decay = λ·‖θ‖₂²; adds L2 regularization to the loss.
9. ⚠️ Never evaluate a model with model.train(); BatchNorm & Dropout will corrupt results.
10. ⚠️ ImageNet mean/std ≈ [0.485, 0.456, 0.406] / [0.229, 0.224, 0.225]; mismatched normalization kills pretrained weights.