Cloud ML - AWS Certified Machine Learning Engineer – Associate (MLA-C01): Problem Framing (Classification, Regression, Clustering, Anomaly Detection)

AWS_ML – Problem Framing (Classification, Regression, Clustering, Anomaly Detection)

AWS Certified Machine Learning – Specialty: Problem Framing Study Guide

(Classification, Regression, Clustering, Anomaly Detection)

What This Is

Problem framing is the first and most critical step in any ML project—it defines what you’re solving, how you’ll measure success, and which algorithm to use. In AWS, this means mapping business problems (e.g., "Why are customers churning?") to ML tasks (e.g., binary classification with Amazon SageMaker Autopilot or XGBoost). Real-world example: A fintech company uses SageMaker to frame fraud detection as a binary classification problem (fraud vs. not fraud), then deploys the model behind an API Gateway + Lambda endpoint for real-time scoring.

Key Terms & Services

Core ML Problem Types

• Classification (Binary/Multi-class):
  Predicts a discrete label (e.g., spam/not spam, customer churn risk: high/medium/low).
  AWS tools: SageMaker Autopilot, XGBoost, BlazingText (text classification).
  Use case: Flagging fraudulent transactions in Amazon Fraud Detector.

• Regression:
  Predicts a continuous value (e.g., house price, sales forecast).
  AWS tools: SageMaker Linear Learner, Factorization Machines, DeepAR (time-series).
  Use case: Forecasting demand for AWS Supply Chain.

• Clustering:
  Groups unlabeled data by similarity (e.g., customer segmentation, anomaly detection).
  AWS tools: SageMaker K-Means, SageMaker Object2Vec (for embeddings).
  Use case: Segmenting users for targeted marketing in Amazon Personalize.

• Anomaly Detection:
  Identifies rare or unexpected patterns (e.g., fraud, equipment failure).
  AWS tools: SageMaker Random Cut Forest (RCF), Amazon Lookout for Metrics, Amazon DevOps Guru.
  Use case: Detecting server failures in AWS CloudWatch Anomaly Detection.

AWS Services for Problem Framing

• Amazon SageMaker Clarify:
  Detects bias and feature importance during problem framing (e.g., "Is our churn model biased against a demographic?").
  Key feature: Generates bias reports for protected groups (e.g., gender, age).

• SageMaker Processing:
  Runs preprocessing jobs (e.g., feature engineering, data validation) before training.
  Use case: Cleaning raw logs from Amazon Kinesis Data Firehose before clustering.

• SageMaker Autopilot:
  Automates algorithm selection and hyperparameter tuning for classification/regression.
  Limitation: Only supports tabular data (not images/text).

• Amazon Fraud Detector:
  Pre-built binary classification service for fraud detection (uses XGBoost under the hood).
  Use case: Real-time transaction scoring with Amazon EventBridge triggers.

• Amazon Lookout for Equipment:
  Anomaly detection for industrial sensors (e.g., predicting pump failures).
  Key feature: No ML expertise required—just upload sensor data.

• AWS Glue DataBrew:
  Visual data cleaning for problem framing (e.g., handling missing values before regression).
  Alternative: SageMaker Data Wrangler (more ML-focused).

Step-by-Step: Framing an ML Problem in AWS

1. Define the Business Problem

• Action: Translate the business goal into an ML task.
  Example:
• Business goal: "Reduce customer churn by 20%."
• ML task: Binary classification (churn = 1, no churn = 0).
• Success metric: Precision/Recall (avoid false positives in retention offers).

2. Explore & Validate Data

• Action: Use SageMaker Studio or AWS Glue DataBrew to:
• Check for missing values (e.g., 30% of "customer tenure" data is null).
• Identify class imbalance (e.g., 95% non-churners, 5% churners).
• Detect bias with SageMaker Clarify (e.g., "Does the model perform worse for users in Region X?").

3. Select the ML Approach

• Rule of thumb:
• Classification: Predict a category (e.g., fraud, spam).
• Regression: Predict a number (e.g., price, demand).
• Clustering: Group similar items (e.g., customer segments).
• Anomaly Detection: Flag outliers (e.g., fraud, failures).
• AWS shortcuts:
• Use SageMaker Autopilot for tabular data (auto-selects algorithm).
• Use Amazon Fraud Detector for pre-built fraud models.
• Use Lookout for Metrics for time-series anomalies.

4. Engineer Features

• Action: Create features in SageMaker Processing or AWS Glue:
• Example features for churn prediction:
  • avg_session_duration (from Amazon Athena queries on S3 logs).
  • days_since_last_purchase (from DynamoDB).
• Store features in SageMaker Feature Store for reuse.

5. Train & Evaluate

• Action: Train a model in SageMaker and evaluate:
• Classification: Use F1-score (balance of precision/recall) or AUC-ROC.
• Regression: Use RMSE or R².
• Clustering: Use Silhouette Score (how similar items are to their own cluster).
• Anomaly Detection: Use Precision@K (e.g., "Top 100 anomalies contain 90% of true fraud").

6. Deploy & Monitor

• Action: Deploy to SageMaker Endpoints or Lambda (for serverless):
• Example: Deploy a churn model to SageMaker and trigger SNS alerts for high-risk users.
• Monitor drift with SageMaker Model Monitor (e.g., "Feature distributions changed—retrain!").

Common Mistakes

Certification Exam Insights

1. Tricky Service-Selection Traps

• Fraud Detection:
• Trap: Candidates pick SageMaker for fraud detection by default.
• Correct: Amazon Fraud Detector is the managed service for fraud (pre-built models, lower effort).

• When to use SageMaker: If you need custom features or multi-class fraud types.

• Anomaly Detection:

• Trap: Candidates confuse SageMaker Random Cut Forest (RCF) with Amazon Lookout for Metrics.

• Correct:

  
  
  • Use RCF for static data (e.g., server logs).
  • Use Lookout for Metrics for time-series (e.g., sales spikes).

• Clustering vs. Classification:

• Trap: Candidates use K-Means for labeled data (e.g., "cluster customers by churn risk").
• Correct: If labels exist, use classification (e.g., XGBoost). K-Means is for unlabeled data.

2. Key Constraints

• SageMaker Autopilot:
• Only supports CSV/Parquet (no JSON, images, or text).
• Max 100 features (exam loves this limit!).
• Amazon Fraud Detector:
• Requires historical fraud labels (can’t use for new fraud types).
• SageMaker Clarify:
• Only detects bias in tabular data (not images/text).

3. "Which Service?" Scenarios

• Scenario: "A retail company wants to segment customers into groups for targeted ads. They have no labels."
• Answer: SageMaker K-Means (clustering for unlabeled data).

• Distractor: SageMaker Autopilot (requires labels).

• Scenario: "A bank needs to predict loan defaults (yes/no) with high precision."

• Answer: SageMaker XGBoost (binary classification with F1-score optimization).
• Distractor: SageMaker Linear Learner (better for regression).

Quick Check Questions

1.

A healthcare startup wants to predict patient readmission risk (high/medium/low) using EHR data. Which AWS service should they use for the core ML task? ✅ SageMaker Autopilot (multi-class classification for tabular data).
❌ Amazon Comprehend Medical (NLP for text, not structured EHR data).
❌ SageMaker K-Means (clustering, not classification).

2.

A manufacturing plant wants to detect equipment failures in real time using sensor data. The data has no labels. Which AWS service is the best fit? ✅ Amazon Lookout for Equipment (anomaly detection for unlabeled sensor data).
❌ SageMaker Random Cut Forest (works but requires manual setup).
❌ Amazon Fraud Detector (for fraud, not equipment failures).

3.

A marketing team wants to forecast quarterly sales (a continuous number) using historical data. Which metric should they optimize for? ✅ RMSE (Root Mean Squared Error) (standard for regression).
❌ F1-score (for classification).
❌ Silhouette Score (for clustering).

Last-Minute Cram Sheet

1. Classification = Predict categories (e.g., fraud, spam). Use SageMaker XGBoost or Autopilot.
2. Regression = Predict numbers (e.g., price, demand). Use SageMaker Linear Learner.
3. Clustering = Group unlabeled data (e.g., customer segments). Use SageMaker K-Means.
4. Anomaly Detection = Flag outliers (e.g., fraud, failures). Use Lookout for Metrics (time-series) or RCF (static data).
5. ⚠️ SageMaker Autopilot only works for tabular data (no images/text).
6. Amazon Fraud Detector = Pre-built binary classification for fraud (no ML expertise needed).
7. SageMaker Clarify = Detects bias in tabular data (not images/text).
8. Lookout for Equipment = Anomaly detection for industrial sensors (no labels needed).
9. ⚠️ K-Means is for unlabeled data—use classification if labels exist.
10. F1-score = Best for imbalanced classification (e.g., fraud detection). AUC-ROC = Best for balanced data.