Intro to Business Statistics: Time Series Analysis - Measuring Accuracy MAE MSE RMSE, MAPE

What This Is

Measuring accuracy is crucial in business decisions, as it helps evaluate the performance of forecasting models, quality control processes, and marketing strategies. A retail chain wants to know if its average daily sales exceed $10,000 to determine if it needs to adjust inventory levels. To do this, it uses metrics like Mean Absolute Error (MAE), Mean Squared Error (MSE), Root Mean Squared Error (RMSE), and Mean Absolute Percentage Error (MAPE).

Key Formulas & Symbols

• MAE = (1/n) * ?|y_i - y?_i| where y_i = actual value, y?_i = predicted value, n = number of observations.
• MSE = (1/n) * ?(y_i - y?_i)^2 where y_i = actual value, y?_i = predicted value, n = number of observations.
• RMSE = ?(MSE) where MSE = Mean Squared Error.
• MAPE = (1/n) * ?|((y_i - y?_i) / y_i) * 100| where y_i = actual value, y?_i = predicted value, n = number of observations.
• y?_i = ?_0 + ?_1x_i where y?_i = predicted value, ?_0 = intercept, ?_1 = slope, x_i = independent variable.
• R^2 = 1 - (?(y_i - y?_i)^2 / ?(y_i - y?)^2) where y_i = actual value, y?_i = predicted value, y? = mean of actual values.

Step-by-Step Procedure

1. Define the problem: Identify the business question and the relevant data.
2. Choose the metric: Select the appropriate accuracy metric (MAE, MSE, RMSE, MAPE) based on the business question and data.
3. Compute the metric: Calculate the chosen metric using the relevant formulas.
4. Interpret the results: Compare the calculated metric to a benchmark or a target value to determine if the business question is answered.
5. Consider additional metrics: Calculate additional metrics like R^2 to evaluate the goodness of fit of the model.

Common Mistakes

• Mistake: Using MAE when the actual values are in different units.
• Correction: Use a metric like MAPE that takes into account the unit of measurement.
• Mistake: Failing to check for outliers in the data.
• Correction: Use robust metrics like RMSE that are less affected by outliers.
• Mistake: Misinterpreting the p-value as the probability that the null hypothesis is true.
• Correction: Understand that the p-value is the probability of observing the data (or more extreme) if the null hypothesis is true.

Quick Practice Problems

1. A company wants to evaluate the accuracy of its sales forecasting model. The actual sales are $100,000, and the predicted sales are $90,000. What is the MAE? Answer: $5,000. The MAE is calculated as the average absolute difference between the actual and predicted values.
2. A quality control process has a mean defect rate of 5%. The actual defect rate is 10%. What is the MAPE? Answer: 100%. The MAPE is calculated as the average absolute percentage difference between the actual and predicted defect rates.
3. A marketing campaign has a predicted return on investment (ROI) of 20%. The actual ROI is 15%. What is the RMSE? Answer: 5%. The RMSE is calculated as the square root of the MSE, which is the average squared difference between the actual and predicted ROIs.

Last-Minute Cram Sheet

1. MAE = (1/n) * ?|y_i - y?_i|.
2. MSE = (1/n) * ?(y_i - y?_i)^2.
3. RMSE = ?(MSE).
4. MAPE = (1/n) * ?|((y_i - y?_i) / y_i) * 100|.
5. R^2 = 1 - (?(y_i - y?_i)^2 / ?(y_i - y?)^2).
6. ? = 0.05 (default significance level).
7. p-value is NOT the probability that H? is true – it’s the probability of observing the data (or more extreme) if H? is true .
8. Use robust metrics like RMSE when dealing with outliers.
9. MAE is sensitive to outliers, while MAPE is not.
10. RMSE is the square root of MSE.