Business Analytics 101: Machine Learning Fundamentals Classification Algorithms kNearest Neighbors Decision Trees Random Forest Naive Bayes Support Vector Machines

What This Is

Classification algorithms are a set of statistical and machine learning techniques used to predict a categorical outcome based on input features. In business analytics, classification algorithms are crucial for applications such as customer segmentation, credit risk assessment, and fraud detection. For instance, a bank might use a classification algorithm to predict whether a customer is likely to default on a loan based on their credit history, income, and other factors.

Key Formulas & Metrics

• Accuracy = (TP + TN) / (TP + TN + FP + FN) – proportion of correctly classified instances.
• TP: True Positives (correctly predicted positive instances)
• TN: True Negatives (correctly predicted negative instances)
• FP: False Positives (incorrectly predicted positive instances)

• FN: False Negatives (incorrectly predicted negative instances)

• Precision = TP / (TP + FP) – proportion of true positives among all predicted positive instances.

• Recall = TP / (TP + FN) – proportion of true positives among all actual positive instances.
• F1 Score = 2 * (Precision * Recall) / (Precision + Recall) – harmonic mean of precision and recall.
• Area Under the ROC Curve (AUC) = 0.5 + (1/π) * ∫[0,1] (dFPR/dFPR) dFPR – measure of model's ability to distinguish between classes.
• Mean Absolute Error (MAE) = (1/n) Σ|yᵢ − ŷᵢ| – average absolute forecast error.
• R² = 1 − (SS_res / SS_tot) – proportion of variance explained by the regression model.

Step-by-Step Procedure

1. Data Preparation: Collect and preprocess the data, handling missing values and outliers.
2. Feature Selection: Select relevant features that contribute to the classification problem.
3. Model Selection: Choose a suitable classification algorithm (e.g., k-Nearest Neighbors, Decision Trees, Random Forest, Naive Bayes, Support Vector Machines).
4. Model Training: Train the model using the selected algorithm and features.
5. Model Evaluation: Evaluate the model's performance using metrics such as accuracy, precision, recall, and F1 score.
6. Model Deployment: Deploy the trained model in a production environment.

Common Mistakes

1. Mistake: Confusing correlation with causation.
2. Correction: Establish a clear cause-and-effect relationship between variables.
3. Mistake: Misinterpreting p-values.
4. Correction: Understand that p-values indicate the probability of observing the data (or more extreme) if the null hypothesis is true.
5. Mistake: Using the wrong error metric for a business problem.
6. Correction: Choose an error metric that aligns with the business objective (e.g., mean squared error for regression, accuracy for classification).

Software / Tool Tips

• Python with scikit-learn: Use the train_test_split function for data splitting, DecisionTreeClassifier for decision trees, and RandomForestClassifier for random forests.
• R: Use the train function for model training, predict for predictions, and confusionMatrix for model evaluation.
• Tableau: Use the Classification node for building classification models and the Scorecard node for model evaluation.

Quick Practice Problem

Problem: Compute the F1 score for a classification model with precision = 0.8 and recall = 0.7.

Answer: F1 score = 2 * (0.8 * 0.7) / (0.8 + 0.7) = 0.7143.

Last-Minute Cram Sheet

1. Accuracy = (TP + TN) / (TP + TN + FP + FN) – proportion of correctly classified instances.
2. Precision = TP / (TP + FP) – proportion of true positives among all predicted positive instances.
3. Recall = TP / (TP + FN) – proportion of true positives among all actual positive instances.
4. F1 Score = 2 * (Precision * Recall) / (Precision + Recall) – harmonic mean of precision and recall.
5. AUC = 0.5 + (1/π) * ∫[0,1] (dFPR/dFPR) dFPR – measure of model's ability to distinguish between classes.
6. MAE = (1/n) Σ|yᵢ − ŷᵢ| – average absolute forecast error.
7. R² = 1 − (SS_res / SS_tot) – proportion of variance explained by the regression model.
8. p-value is NOT the probability that H₀ is true – it’s the probability of observing the data (or more extreme) if H₀ is true ⚠️
9. Correlation does NOT imply causation ⚠️
10. Always choose the error metric that aligns with the business objective