Business Analytics 101: Statistical Inference Hypothesis Testing Null vs Alternative ttest Chisquare ANOVA pvalues Type I and Type II Errors Power

What This Is

Hypothesis testing is a statistical method used to determine whether there is enough evidence to support a claim or hypothesis about a population based on a sample of data. In business analytics, hypothesis testing is crucial for making informed decisions, such as determining whether a new marketing campaign is effective, whether a product is preferred by a specific demographic, or whether a process improvement has a significant impact on productivity. For example, a company wants to determine whether the average salary of its employees in the sales department is higher than the average salary of employees in the marketing department.

Key Formulas & Metrics

• Null Hypothesis (H₀) = μ = μ₀ – the default assumption that there is no effect or no difference.
• Alternative Hypothesis (H₁) = μ ≠ μ₀ – the hypothesis that there is an effect or a difference.
• t-statistic = (x̄ - μ₀) / (s / √n) – a measure of how many standard errors the sample mean is away from the population mean.
• p-value = P(t > |t-statistic|) – the probability of observing a t-statistic at least as extreme as the one observed, assuming H₀ is true.
• Type I Error (α) = P(reject H₀ | H₀ is true) – the probability of rejecting a true null hypothesis.
• Type II Error (β) = P(fail to reject H₀ | H₁ is true) – the probability of failing to reject a false null hypothesis.
• Power = 1 - β – the probability of rejecting a false null hypothesis.
• Chi-square statistic = Σ [(observed - expected)^2 / expected] – a measure of how much the observed frequencies differ from the expected frequencies.
• Degrees of Freedom (df) = n - 1 – the number of independent pieces of information in a dataset.
• F-statistic = (MS_between / MS_within) – a measure of the ratio of the variance between groups to the variance within groups.

Step-by-Step Procedure

1. Formulate the null and alternative hypotheses: Clearly define the null and alternative hypotheses based on the research question.
2. Choose a significance level (α): Select a significance level (e.g., 0.05) that determines the maximum probability of a Type I error.
3. Calculate the test statistic: Use the data to calculate the test statistic (e.g., t-statistic, Chi-square statistic, F-statistic).
4. Determine the p-value: Calculate the p-value associated with the test statistic, which represents the probability of observing the data (or more extreme) if the null hypothesis is true.
5. Interpret the results: Compare the p-value to the significance level (α) and make a decision about the null hypothesis.

Common Mistakes

• Mistake: Confusing correlation with causation.
• Correction: Correlation does not imply causation; there may be other factors at play.
• Mistake: Misinterpreting p-values.
• Correction: A low p-value does not necessarily mean that the null hypothesis is true; it only indicates that the observed data is unlikely if the null hypothesis is true.
• Mistake: Using the wrong error metric for a business problem.
• Correction: Use the error metric that is most relevant to the business problem (e.g., mean absolute error for forecasting, mean squared error for regression).

Software / Tool Tips

• Python with scikit-learn: Use the ttest_ind() function for independent samples t-tests and the chi2_contingency() function for Chi-square tests.
• R: Use the t.test() function for t-tests and the chisq.test() function for Chi-square tests.
• Excel: Use the T.TEST() function for t-tests and the CHISQ.TEST() function for Chi-square tests.

Quick Practice Problem

A company wants to determine whether the average salary of its employees in the sales department is higher than the average salary of employees in the marketing department. The sample mean salary for the sales department is $80,000, and the sample mean salary for the marketing department is $70,000. The sample standard deviation for both departments is $10,000. What is the p-value associated with a two-sample t-test?

Answer: The p-value is approximately 0.01.

Last-Minute Cram Sheet

• Null Hypothesis (H₀) = μ = μ₀: the default assumption that there is no effect or no difference.
• Alternative Hypothesis (H₁) = μ ≠ μ₀: the hypothesis that there is an effect or a difference.
• t-statistic = (x̄ - μ₀) / (s / √n): a measure of how many standard errors the sample mean is away from the population mean.
• p-value = P(t > |t-statistic|): the probability of observing a t-statistic at least as extreme as the one observed, assuming H₀ is true.
• Type I Error (α) = P(reject H₀ | H₀ is true): the probability of rejecting a true null hypothesis.
• Type II Error (β) = P(fail to reject H₀ | H₁ is true): the probability of failing to reject a false null hypothesis.
• Power = 1 - β: the probability of rejecting a false null hypothesis.
• Chi-square statistic = Σ [(observed - expected)^2 / expected]: a measure of how much the observed frequencies differ from the expected frequencies.
• Degrees of Freedom (df) = n - 1: the number of independent pieces of information in a dataset.
• F-statistic = (MS_between / MS_within): a measure of the ratio of the variance between groups to the variance within groups.
• ⚠️ p-value is NOT the probability that H₀ is true – it’s the probability of observing the data (or more extreme) if H₀ is true.