Data Science and Machine Learning 101: Foundations and Math Statistics for Data Science Descriptive vs Inferential Hypothesis Testing pvalues Confidence Intervals

What This Is

Statistics for Data Science is the toolbox that lets you summarize what your data look like (descriptive statistics) and draw conclusions about a larger population or future observations (inferential statistics). In a churn‑prediction project, you first describe the current customers (average tenure, churn rate) and then test whether a new marketing campaign actually reduces churn, using hypothesis testing, p‑values, and confidence intervals. Without these tools you cannot quantify uncertainty, compare models rigorously, or convince stakeholders that your results are real and not random noise.

Key Terms & Formulas

• Descriptive Statistics – Numbers that characterise a dataset (mean, median, variance, skewness). Used for quick data‑level insight before modeling.
• Inferential Statistics – Methods that infer properties of a population from a sample (confidence intervals, hypothesis tests). Enables you to generalise beyond the data you have.
• Sample Mean ( (\bar{x}) ) – (\displaystyle \bar{x} = \frac{1}{n}\sum_{i=1}^{n} x_i). The “average” of a sample; foundation for many estimators.
• Sample Standard Deviation ( (s) ) – (\displaystyle s = \sqrt{\frac{1}{n-1}\sum_{i=1}^{n}(x_i-\bar{x})^2}). Measures spread; denominator (n-1) gives an unbiased estimator.
• Standard Error ( (SE) ) – (\displaystyle SE = \frac{s}{\sqrt{n}}). The expected variability of the sample mean if you repeatedly drew samples.
• Confidence Interval (CI) – (\displaystyle \bar{x} \pm t_{\alpha/2,\,df}\,SE). Gives a range that, say 95% of the time, will contain the true population mean. (t_{\alpha/2,df}) is the critical t‑value.
• Null Hypothesis ((H_0)) – The default claim (e.g., “the new email campaign does not change churn”). Tested against an alternative.
• Alternative Hypothesis ((H_A)) – The claim you hope to support (e.g., “the campaign reduces churn”).
• Test Statistic – A standardized value (z, t, χ², etc.) that quantifies the distance between observed data and (H_0).
• p‑value – (\displaystyle p = P(\text{Test statistic} \ge \text{observed} \mid H_0)). Probability of seeing data at least as extreme as yours if (H_0) were true. Small p ⇒ reject (H_0).
• Type I Error (α) – Rejecting a true (H_0); the false‑positive rate you set (commonly 0.05).
• Type II Error (β) – Failing to reject a false (H_0); related to power = (1-β).
• Two‑Sample t‑test (independent) – Tests whether two groups have different means:
  [ t = \frac{\bar{x}_1-\bar{x}_2}{\sqrt{s_1^2/n_1 + s_2^2/n_2}} ]
  Use when you compare churn before vs. after a campaign.

Step‑by‑Step / Process Flow

1. Load & Clean – df = pd.read_csv('churn.csv'); df.dropna(inplace=True)
2. Explore Descriptively – df['tenure'].describe(); df['tenure'].hist(); compute mean, std, and plot box‑plots to spot outliers.
3. Formulate Hypotheses – Example:
4. (H_0): Mean churn rate after campaign = mean churn rate before.
5. (H_A): Mean churn rate after campaign < before.
6. Choose the Right Test – If the two churn samples are independent and roughly normal → two‑sample t‑test; otherwise use Mann‑Whitney U.
7. Compute Test Statistic & p‑value –
   python
from scipy.stats import ttest_ind
t, p = ttest_ind(churn_before, churn_after, equal_var=False)
8. Interpret & Report – If p < 0.05, reject (H_0); also present a 95 % CI for the difference in means. Communicate practical impact (e.g., “the campaign cuts churn by 1.2 % ± 0.4 %”).

Common Mistakes

• Mistake: Treating a p‑value as the probability that (H_0) is true.
  Correction: p‑value is conditional on (H_0) being true; it tells you how surprising your data are, not the truth of the hypothesis.

• Mistake: Using the same data to both choose a test and report the p‑value (data‑dredging).
  Correction: Pre‑specify the test (or split data into a “exploratory” and “confirmatory” set) to avoid inflated Type I error.

• Mistake: Ignoring the effect size and focusing only on statistical significance.
  Correction: Report the magnitude (e.g., mean difference, Cohen’s d) alongside p‑value; a tiny p‑value can accompany a negligible practical impact.

• Mistake: Assuming normality without checking.
  Correction: Visualise with Q‑Q plots or run a Shapiro‑Wilk test; switch to non‑parametric tests if normality fails.

• Mistake: Using a 95 % CI when the underlying distribution is heavily skewed.
  Correction: Consider a bootstrap CI (np.random.choice resampling) which does not rely on normality.

Data Science Interview / Practical Insights

1. “Explain the difference between a confidence interval and a prediction interval.” – CI covers the population parameter; a prediction interval covers a future individual observation.
2. “When would you prefer a non‑parametric test over a t‑test?” – When the sample size is small (<30) or the data are clearly non‑normal or have outliers.
3. “What does a p‑value of 0.07 mean in a business context?” – It indicates insufficient evidence at the 5 % level, but you may still consider the effect if the cost of a Type II error is high.
4. “How do you control the family‑wise error rate when testing many features?” – Use Bonferroni correction (α_adj = α / m) or a false‑discovery‑rate method like Benjamini‑Hochberg.

Quick Check Questions

1. Scenario: You run a two‑sample t‑test on conversion rates for two landing pages and obtain p = 0.03.
   Answer: Reject (H_0) at α = 0.05; the difference is statistically significant.

2. Scenario: Your 95 % CI for the lift of a new recommendation algorithm is ([-0.2\%, +1.5\%]).
   Answer: Because the interval includes 0, you cannot claim a statistically significant improvement.

3. Scenario: After a pilot, the churn rate dropped from 8 % to 6.5 % (n = 200 each). The t‑test p‑value is 0.12, but the business impact is large.
   Answer: Consider power (β) – the sample may be too small; a larger test or a Bayesian approach could be more informative.

Last‑Minute Cram Sheet (10 one‑liners)

1. Mean vs. Median: Mean is sensitive to outliers; median is robust.
2. Standard Error = s / √n – tells how precisely you know the sample mean.
3. 95 % CI ≈ mean ± 1.96·SE for large n (z‑approximation).
4. p‑value < α ⇒ reject (H_0); otherwise fail to reject.
5. Effect size matters: Cohen’s d = (Δ mean) / pooled s.
6. Two‑sample t‑test assumes: independent samples, approx. normal, equal variances (use Welch’s if not).
7. Bootstrap CI: Resample with replacement → compute statistic → take percentiles (e.g., 2.5 % & 97.5 %).
8. Bonferroni correction: α_adj = α / #tests; controls family‑wise error.
9. Confidence vs. Prediction: CI → parameter; PI → future observation.
10. ⚠️ p‑value ≠ “probability of hypothesis”; it’s the probability of data given the hypothesis.

Use this guide to turn raw numbers into trustworthy stories and to survive any statistics‑heavy interview. Good luck!