Intro to Business Statistics: Correlation and Regression Least Squares Method Calculating b₀ and b₁

What This Is

The Least Squares Method is a statistical technique used to estimate the relationship between two continuous variables, often denoted as X (independent variable) and Y (dependent variable). This method is crucial in business decisions, such as predicting sales based on advertising expenses or determining the impact of interest rates on loan defaults. For instance, a retail chain wants to know if average daily sales exceed $10,000 when the advertising budget is $5,000.

Key Formulas & Symbols

• b₁ = Σ[(xi - x̄)(yi - ȳ)] / Σ(xi - x̄)² where xi = individual data points, x̄ = sample mean, yi = individual data points, ȳ = sample mean.
• b₀ = ȳ - b₁x̄ where x̄ = sample mean.
• r² = [Σ(xi - x̄)(yi - ȳ)]² / [Σ(xi - x̄)² * Σ(yi - ȳ)²] where xi = individual data points, x̄ = sample mean, yi = individual data points, ȳ = sample mean.
• SSE = Σ(yi - (b₀ + b₁xi))² where yi = individual data points, b₀ = intercept, b₁ = slope.
• SST = Σ(yi - ȳ)² where yi = individual data points, ȳ = sample mean.
• SSE / SST where SSE = sum of squared errors, SST = total sum of squares.
• R² = 1 - (SSE / SST) where R² = coefficient of determination.
• F = (b₁² * (n - 2)) / (SSE / (n - 2)) where b₁ = slope, n = sample size, SSE = sum of squared errors.
• F-statistic = (b₁² * (n - 2)) / (SSE / (n - 2)) where b₁ = slope, n = sample size, SSE = sum of squared errors.

Step-by-Step Procedure

1. State hypotheses: Formulate null and alternative hypotheses (e.g., H₀: b₁ = 0, H₁: b₁ ≠ 0).
2. Choose test: Select the appropriate test (e.g., simple linear regression, multiple linear regression).
3. Compute test statistic: Calculate the F-statistic or t-statistic using the given formulas.
4. Find p-value or critical value: Determine the p-value associated with the calculated test statistic or find the critical value from the F-distribution table.
5. Compare to α: Compare the p-value or critical value to the chosen significance level (α = 0.05).
6. Conclude: Based on the comparison, reject or fail to reject the null hypothesis.

Common Mistakes

• Mistake: Misinterpreting the p-value as the probability that the null hypothesis is true.
• Correction: The p-value is the probability of observing the data (or more extreme) if the null hypothesis is true. It does not provide information about the probability of the null hypothesis being true.
• Mistake: Failing to check for multicollinearity among independent variables.
• Correction: Multicollinearity can lead to unstable estimates of regression coefficients and should be checked before proceeding with the analysis.
• Mistake: Ignoring the assumption of normality of residuals.
• Correction: Non-normal residuals can lead to incorrect conclusions and should be checked using diagnostic plots or tests.

Quick Practice Problems

1. A marketing firm wants to know if the number of social media followers (X) is related to the number of sales (Y). The sample mean of X is 1000, and the sample mean of Y is 500. If the slope (b₁) is 0.05, what is the intercept (b₀)?

b₀ = ȳ - b₁x̄ = 500 - 0.05(1000) = 450

1. A company wants to determine if the interest rate (X) affects the loan default rate (Y). The sample size is 20, and the sum of squared errors (SSE) is 100. If the total sum of squares (SST) is 500, what is the coefficient of determination (R²)?

R² = 1 - (SSE / SST) = 1 - (100 / 500) = 0.8

1. A retail chain wants to know if the advertising budget (X) affects the sales (Y). The sample mean of X is $5,000, and the sample mean of Y is $10,000. If the slope (b₁) is 0.02, what is the p-value associated with the F-statistic?

F = (b₁² * (n - 2)) / (SSE / (n - 2)) = (0.02² * (20 - 2)) / (100 / (20 - 2)) = 0.16. The p-value associated with this F-statistic is approximately 0.7.

Last-Minute Cram Sheet

1. F-statistic = (b₁² * (n - 2)) / (SSE / (n - 2)) where b₁ = slope, n = sample size, SSE = sum of squared errors.
2. R² = 1 - (SSE / SST) where R² = coefficient of determination, SSE = sum of squared errors, SST = total sum of squares.
3. b₁ = Σ[(xi - x̄)(yi - ȳ)] / Σ(xi - x̄)² where xi = individual data points, x̄ = sample mean, yi = individual data points, ȳ = sample mean.
4. b₀ = ȳ - b₁x̄ where x̄ = sample mean.
5. SSE = Σ(yi - (b₀ + b₁xi))² where yi = individual data points, b₀ = intercept, b₁ = slope.
6. SST = Σ(yi - ȳ)² where yi = individual data points, ȳ = sample mean.
7. p-value is NOT the probability that H₀ is true – it’s the probability of observing the data (or more extreme) if H₀ is true.
8. ⚠️ Multicollinearity can lead to unstable estimates of regression coefficients.
9. Assumptions of linear regression: linearity, independence, homoscedasticity, normality of residuals, and no multicollinearity.
10. F-distribution table is used to find critical values for F-statistic.