Data Science and Machine Learning 101: Model Deployment and MLOps Model Governance and Responsible AI Bias Detection Explainability LIME SHAP Fairness Metrics

What This Is

Model Governance and Responsible AI are the policies, tools, and practices that keep machine‑learning systems trustworthy, fair, and transparent. In a production pipeline you must detect bias, explain predictions, and measure fairness before you ship a model—otherwise you risk regulatory penalties, brand damage, or harmful decisions (e.g., a churn‑prediction model that systematically downgrades minority customers).

Key Terms & Formulas

• Bias (statistical) = E[ ŷ ] – y – systematic error of a model’s predictions (different from “bias” in fairness).
• Statistical Parity Difference (SPD) = P( ŷ = 1 | A = 1 ) – P( ŷ = 1 | A = 0 ) – measures disparity between a privileged (A=1) and unprivileged (A=0) group.
• Equal Opportunity Difference (EOD) = TPR_A=1 – TPR_A=0 – difference in true‑positive rates across groups.
• Disparate Impact (DI) = P( ŷ = 1 | A = 1 ) / P( ŷ = 1 | A = 0 ) – ratio; DI < 0.8 often flags illegal bias (US EEOC “80 % rule”).
• LIME (Local Interpretable Model‑agnostic Explanations) – fits a weighted linear model g around a single instance x:

[ \underset{g \in G}{\arg\min}\; L\big(f, g, \pi_x\big) + \Omega(g) ]

where πₓ weights perturbed samples by proximity to x.
- SHAP (SHapley Additive exPlanations) – assigns each feature i a contribution ϕᵢ based on Shapley values:

[ \phi_i = \sum_{S \subseteq N\setminus{i}} \frac{|S|!\,(|N|-|S|-1)!}{|N|!}\,\big[ f_{S\cup{i}}(x_{S\cup{i}}) - f_S(x_S) \big] ]

(N = set of all features).
- Counterfactual Explanation – “What minimal change to input x would flip the prediction?” often solved by optimization:

[ \min_{\Delta x}\; |\Delta x|p \quad \text{s.t.}\; f(x+\Delta x)=y ]}
- Fairness‑aware loss – e.g., Demographic Parity regularization:

[ \mathcal{L}{\text{fair}} = \mathcal{L} | ] }} + \lambda \,| \text{SPD

λ controls the trade‑off between accuracy and fairness.
- Calibration Curve – plots observed frequency vs. predicted probability; mis‑calibration can hide bias.
- Model Card – a concise documentation template (model name, intended use, metrics, ethical considerations) that accompanies any released model.

Step‑by‑Step / Process Flow

1. Load & Split – X, y, A = load_data(); X_train, X_test, y_train, y_test, A_train, A_test = train_test_split(..., stratify=y) (A = protected attribute).
2. Baseline Training – Fit a quick model (e.g., LogisticRegression) and record standard metrics (accuracy, ROC‑AUC).
3. Bias Audit – Compute fairness metrics on train and test (SPD, EOD, DI). Flag any that exceed your policy threshold (e.g., |SPD| > 0.1).
4. Explainability Pass – For a handful of high‑impact predictions, run lime.lime_tabular.LimeTabularExplainer and shap.TreeExplainer. Store the top‑k feature contributions.
5. Mitigation Loop – If bias is detected, choose a mitigation:
6. Pre‑processing (reweighing, aif360 Reweighing).
7. In‑processing (add fairness regularizer to loss).
8. Post‑processing (threshold adjustment per group).
   
   Retrain and re‑evaluate until metrics satisfy both accuracy and fairness constraints.
9. Governance Packaging – Generate a Model Card (Markdown) and a Data Sheet for the training set; version‑control them alongside the model artifact. Deploy with monitoring hooks that recompute fairness metrics on live data weekly.

Common Mistakes

• Mistake: Only checking overall accuracy.
  Correction: Always compute group‑wise metrics; a model can be 90 % accurate overall but 70 % accurate for a minority group.

• Mistake: Treating SHAP values as causal explanations.
  Correction: SHAP quantifies association; it does not prove that changing a feature will change the outcome. Use counterfactuals for causal “what‑if” analysis.

• Mistake: Applying LIME on high‑dimensional sparse data without dimensionality reduction.
  Correction: Reduce features (e.g., via TF‑IDF → top‑k) or use LimeTabularExplainer with feature_selection='auto' to keep explanations stable.

• Mistake: Hard‑coding a single fairness threshold for all projects.
  Correction: Align thresholds with domain risk (e.g., stricter for credit scoring vs. movie recommendation) and document the rationale.

• Mistake: Skipping calibration before fairness checks.
  Correction: Poorly calibrated probabilities can inflate disparity metrics; calibrate with CalibratedClassifierCV first.

Data Science Interview / Practical Insights

1. “Explain the difference between demographic parity and equal opportunity.” – Expect you to discuss the output vs. error‑rate focus and when each is appropriate.
2. “How would you use SHAP to debug a model that is unfair to a protected group?” – Talk about aggregating SHAP values per group, spotting features that systematically push predictions down, and then iterating on mitigation.
3. “Why can a model be perfectly calibrated yet still exhibit bias?” – Because calibration aligns predicted probabilities with observed frequencies overall, not across subpopulations.
4. “What is a Model Card and why is it required for regulated industries?” – Highlight transparency, reproducibility, and compliance (e.g., FDA, GDPR).

Quick Check Questions

1. Scenario: Your churn model shows SPD = ‑0.25 (unprivileged group churns less often).
   Answer: The model is biased against the privileged group; you need to reduce the absolute SPD (e.g., reweigh training samples).

2. Scenario: After adding a fairness regularizer, validation loss rises but SPD drops to 0.02.
   Answer: This is the expected accuracy‑fairness trade‑off; decide whether the fairness gain outweighs the loss in predictive performance.

3. Scenario: SHAP summary plot shows “age” has the highest mean absolute contribution, but the feature is not in the data dictionary.
   Answer: Likely a data leakage issue—age may be encoded indirectly (e.g., via zip code); remove or mask it before retraining.

Last‑Minute Cram Sheet (10 one‑liners)

1. SPD = P(ŷ=1|A=1) – P(ŷ=1|A=0) – aim for |SPD| < 0.1 (or per policy).
2. EOD = TPR_A=1 – TPR_A=0 – equal‑opportunity focuses on true‑positive parity.
3. DI = P(ŷ=1|A=1) / P(ŷ=1|A=0) – DI < 0.8 flags potential illegal bias (US “80 % rule”).
4. LIME: Fit a local weighted linear model; use kernel_width ≈ √(num_features) for stable neighborhoods.
5. SHAP: For tree models, shap.TreeExplainer runs in O(N·log M) time (N = samples, M = trees).
6. Counterfactual: Minimize ‖Δx‖₁ for sparse changes; often yields actionable “what‑to‑change” advice.
7. Fairness‑aware loss: Add λ·|SPD| to cross‑entropy; tune λ with a validation set that respects both metrics.
8. Calibration first! – Run CalibratedClassifierCV before fairness checks; mis‑calibrated scores distort disparity numbers.
9. Model Card sections: Intended use, Metrics, Ethical considerations, Limitations, Maintenance plan.
10. ⚠️ Don’t assume “high AUC = fair”; AUC is group‑agnostic and can hide severe subgroup errors.