Intro to Business Statistics: Correlation and Regression Multiple Linear Regression Model Y β₀ β₁X₁ β₂X₂ β kX k ε

What This Is

Multiple Linear Regression (MLR) is a statistical method used to model the relationship between a dependent variable (Y) and one or more independent variables (X₁, X₂, …, X_k). It helps businesses understand how changes in the independent variables affect the dependent variable, enabling informed decision-making. For example, a retail chain wants to know if average daily sales exceed $10,000 based on the number of employees and the number of promotions.

Key Formulas & Symbols

• Y = β₀ + β₁X₁ + β₂X₂ + … + β_kX_k + ε where Y = dependent variable, β₀ = intercept, β₁, β₂, …, β_k = coefficients, X₁, X₂, …, X_k = independent variables, ε = error term.
• R² = 1 - (SSE / SST) where R² = coefficient of determination, SSE = sum of squared errors, SST = total sum of squares.
• F = (MSR / MSE) where F = F-statistic, MSR = mean square regression, MSE = mean square error.
• p-value = P(F > F-statistic) where p-value = probability of observing the data (or more extreme) if H₀ is true, F-statistic = calculated F-statistic.
• β_k = (Cov(X_k, Y) / Var(X_k)) where β_k = coefficient of X_k, Cov(X_k, Y) = covariance between X_k and Y, Var(X_k) = variance of X_k.
• t = (β_k - β_k₀) / (s_e / √(n)) where t = t-statistic, β_k = estimated coefficient, β_k₀ = hypothesized coefficient, s_e = standard error of the estimate, n = sample size.
• R² change = R²_new - R²_old where R² change = change in R², R²_new = R² with additional variable, R²_old = R² without additional variable.
• F-statistic for R² change = (R²_new - R²_old) / (1 - R²_new) / (1 / (n - k - 1)) where F-statistic for R² change = F-statistic for change in R², n = sample size, k = number of independent variables.

Step-by-Step Procedure

1. State hypotheses: Formulate null and alternative hypotheses (e.g., H₀: β_k = 0 vs. H_a: β_k ≠ 0).
2. Choose test: Select the appropriate test statistic (e.g., F-statistic for overall model significance, t-statistic for individual coefficient significance).
3. Compute test statistic: Calculate the test statistic using the given formulas.
4. Find p-value or critical value: Determine the p-value or critical value using a statistical table or software.
5. Compare to α: Compare the p-value or critical value to the significance level (α = 0.05).
6. Conclude: Make a decision based on the comparison (e.g., reject H₀ if p-value < α).

Common Mistakes

• Mistake: Misinterpreting the p-value as the probability that H₀ is true.
• Correction: The p-value is the probability of observing the data (or more extreme) if H₀ is true. It does not provide information about the probability of H₀ being true.
• Mistake: Failing to check for multicollinearity among independent variables.
• Correction: Multicollinearity can lead to unstable estimates and incorrect conclusions. Check for high correlations among independent variables and consider using techniques like principal component analysis or partial least squares regression.
• Mistake: Ignoring the assumption of linearity between the dependent variable and each independent variable.
• Correction: Linearity is a critical assumption in MLR. Check for non-linear relationships using techniques like scatter plots or partial regression plots.

Quick Practice Problems

1. A company wants to know if the number of hours worked per week affects employee productivity. The dependent variable is productivity (Y), and the independent variable is hours worked per week (X). The estimated coefficient for hours worked per week is 0.05, and the standard error of the estimate is 0.01. What is the t-statistic for the coefficient?

t = (0.05 - 0) / (0.01 / √(100)) = 5

The t-statistic is 5, indicating a significant relationship between hours worked per week and productivity.

1. A marketing manager wants to know if the number of social media followers affects sales. The dependent variable is sales (Y), and the independent variable is social media followers (X). The R² for the model with social media followers is 0.3, and the R² without social media followers is 0.2. What is the F-statistic for the change in R²?

F-statistic = (0.3 - 0.2) / (1 - 0.3) / (1 / (100 - 1 - 1)) = 10.67

The F-statistic is 10.67, indicating a significant improvement in the model with social media followers.

1. A quality control manager wants to know if the number of defects per unit affects the cost of production. The dependent variable is cost (Y), and the independent variable is defects per unit (X). The estimated coefficient for defects per unit is 0.01, and the standard error of the estimate is 0.005. What is the t-statistic for the coefficient?

t = (0.01 - 0) / (0.005 / √(50)) = 20

The t-statistic is 20, indicating a significant relationship between defects per unit and cost.

Last-Minute Cram Sheet

1. F-statistic for overall model significance: F = (MSR / MSE) where MSR = mean square regression, MSE = mean square error.
2. t-statistic for individual coefficient significance: t = (β_k - β_k₀) / (s_e / √(n)) where β_k = estimated coefficient, β_k₀ = hypothesized coefficient, s_e = standard error of the estimate, n = sample size.
3. p-value: p-value = P(F > F-statistic) where F-statistic = calculated F-statistic.
4. R²: R² = 1 - (SSE / SST) where SSE = sum of squared errors, SST = total sum of squares.
5. Assumption of linearity: Check for non-linear relationships using techniques like scatter plots or partial regression plots.
6. Assumption of independence: Check for independence using techniques like the Durbin-Watson test.
7. Assumption of homoscedasticity: Check for homoscedasticity using techniques like the Breusch-Pagan test.
8. ⚠️ p-value is NOT the probability that H₀ is true – it’s the probability of observing the data (or more extreme) if H₀ is true.
9. ⚠️ F-statistic is sensitive to sample size – use caution when interpreting results with small sample sizes.
10. ⚠️ R² is not a perfect measure of model fit – use other metrics like AIC or BIC to evaluate model performance.