Intro to Business Statistics: Correlation and Regression Residual Analysis Plots Standardized Residuals Influential Points

What This Is

Residual analysis is a statistical technique used to evaluate the goodness of fit of a regression model by examining the residuals, which are the differences between the observed values and the predicted values. A retail chain wants to know if average daily sales exceed $10,000. They collect data on sales and use regression analysis to model the relationship between sales and advertising expenses. However, they need to check if the model fits the data well and if there are any unusual patterns or outliers.

Key Formulas & Symbols

• Residual (e): The difference between the observed value and the predicted value, e = y - ŷ.
• Standardized Residual (e*): The residual divided by the standard error, e* = e / s.
• Cook's Distance (D): A measure of the influence of a data point on the regression model, D = (e^2 / (p * s^2)) * ((1 / (1 - h_i))^2).
• Leverage (h): A measure of the influence of a data point on the regression model, h = 1 / (n - p).
• DFFITS (DFF): A measure of the influence of a data point on the regression model, DFF = (e / s) * sqrt((1 / (1 - h_i))^2).
• DFBETAS (DFB): A measure of the influence of a data point on the regression coefficients, DFB = (b_i - b_i^0) / sqrt((s^2 / n) * (1 / (1 - h_i))^2).
• Residual Plot: A scatter plot of the residuals against the predicted values.
• Normal Probability Plot: A plot of the residuals against a normal distribution of the same size.
• Influence Function: A measure of the influence of a data point on the regression model.

Step-by-Step Procedure

1. State hypotheses: Formulate the null and alternative hypotheses, e.g., H₀: β₁ = 0 vs. H₁: β₁ ≠ 0.
2. Choose test: Select the appropriate test statistic, e.g., t-statistic for small samples or F-statistic for large samples.
3. Compute test statistic: Calculate the test statistic using the sample data, e.g., t = (b₁ - β₁) / (s * sqrt(1 / (n - p))).
4. Find p-value or critical value: Determine the p-value or critical value using the test statistic and the chosen significance level (α = 0.05).
5. Compare to α: Compare the p-value or critical value to the significance level (α = 0.05) to make a decision.
6. Conclude: Based on the decision, conclude whether to reject the null hypothesis or fail to reject it.

Common Mistakes

• Mistake: Misinterpreting the p-value as the probability that the null hypothesis is true.
• Correction: The p-value is the probability of observing the data (or more extreme) if the null hypothesis is true. It does not provide information about the probability of the null hypothesis being true.
• Mistake: Failing to check for influential points in the data.
• Correction: Influential points can significantly affect the regression model and its conclusions. It is essential to check for influential points using measures such as Cook's Distance, DFFITS, and DFBETAS.
• Mistake: Ignoring the assumptions of the regression model.
• Correction: The assumptions of the regression model, such as linearity, independence, and homoscedasticity, must be checked before drawing conclusions from the model.

Quick Practice Problems

1. A company wants to know if the average daily sales exceed $10,000. They collect data on sales and use regression analysis to model the relationship between sales and advertising expenses. The residual plot shows a non-random pattern. What is the conclusion? Answer: The model does not fit the data well, and there may be unusual patterns or outliers.
2. A marketing firm wants to know if the average response to an advertisement exceeds 100. They collect data on responses and use regression analysis to model the relationship between responses and advertising expenses. The standardized residual plot shows a normal distribution. What is the conclusion? Answer: The model fits the data well, and there are no unusual patterns or outliers.
3. A quality control team wants to know if the average defect rate exceeds 5%. They collect data on defect rates and use regression analysis to model the relationship between defect rates and production levels. The influence function plot shows a data point with high influence. What is the conclusion? Answer: The data point has a significant impact on the regression model, and it may be an outlier or an influential point.

Last-Minute Cram Sheet

1. Residual plot: A scatter plot of the residuals against the predicted values.
2. Normal probability plot: A plot of the residuals against a normal distribution of the same size.
3. Influence function: A measure of the influence of a data point on the regression model.
4. Cook's Distance (D): A measure of the influence of a data point on the regression model, D = (e^2 / (p * s^2)) * ((1 / (1 - h_i))^2).
5. Leverage (h): A measure of the influence of a data point on the regression model, h = 1 / (n - p).
6. DFFITS (DFF): A measure of the influence of a data point on the regression model, DFF = (e / s) * sqrt((1 / (1 - h_i))^2).
7. DFBETAS (DFB): A measure of the influence of a data point on the regression coefficients, DFB = (b_i - b_i^0) / sqrt((s^2 / n) * (1 / (1 - h_i))^2).
8. p-value: The probability of observing the data (or more extreme) if the null hypothesis is true.
9. α = 0.05: The default significance level for hypothesis testing.
10. ⚠️ p-value is NOT the probability that H₀ is true – it’s the probability of observing the data (or more extreme) if H₀ is true.