Analysis of Variance ANOVA Post Hoc Tests (Tukey, Bonferroni)

Concept Summary

• Post hoc tests are statistical procedures used to compare means of multiple groups after an analysis of variance (ANOVA) has been conducted.
• These tests are used to determine which specific groups differ from each other.
• Post hoc tests are also known as multiple comparison tests or pairwise comparison tests.
• The most commonly used post hoc tests are the Tukey's HSD (Honestly Significant Difference) test and the Bonferroni correction.
• Post hoc tests help to avoid the problem of family-wise error rate (FWER) inflation, which occurs when multiple comparisons are made without adjusting for the increased risk of Type I errors.

Questions

WHAT (definitional)

1. What is the purpose of post hoc tests in statistical analysis?
2. Answer: Post hoc tests are used to compare means of multiple groups after an ANOVA has been conducted.
3. Real-world example: In a study on the effect of different fertilizers on plant growth, post hoc tests can be used to compare the means of plant growth in each fertilizer group.
4. Misconception cleared: Post hoc tests are not used to determine the overall significance of an ANOVA, but rather to identify which specific groups differ from each other.
5. What is the difference between the Tukey's HSD test and the Bonferroni correction?
6. Answer: The Tukey's HSD test is a post hoc test that compares all possible pairs of means, while the Bonferroni correction is a method used to adjust the alpha level for multiple comparisons.
7. Real-world example: In a study on the effect of different exercise programs on weight loss, the Tukey's HSD test can be used to compare the means of weight loss in each exercise program group, while the Bonferroni correction can be used to adjust the alpha level for multiple comparisons.
8. Misconception cleared: The Bonferroni correction is not a post hoc test, but rather a method used to adjust the alpha level for multiple comparisons.
9. What is the problem of family-wise error rate (FWER) inflation in multiple comparisons?
10. Answer: FWER inflation occurs when multiple comparisons are made without adjusting for the increased risk of Type I errors.
11. Real-world example: In a study on the effect of different medications on blood pressure, FWER inflation can occur if multiple comparisons are made without adjusting for the increased risk of Type I errors.
12. Misconception cleared: FWER inflation is not a problem that occurs only in post hoc tests, but rather in any situation where multiple comparisons are made without adjusting for the increased risk of Type I errors.

WHY (causal reasoning)

1. Why is it necessary to use post hoc tests after an ANOVA has been conducted?
2. Answer: Post hoc tests are necessary to determine which specific groups differ from each other, and to avoid the problem of FWER inflation.
3. Real-world example: In a study on the effect of different fertilizers on plant growth, post hoc tests can be used to compare the means of plant growth in each fertilizer group, and to determine which fertilizers differ from each other.
4. Misconception cleared: Post hoc tests are not necessary if the ANOVA result is significant, but rather to identify which specific groups differ from each other.
5. Why is the Bonferroni correction used to adjust the alpha level for multiple comparisons?
6. Answer: The Bonferroni correction is used to adjust the alpha level for multiple comparisons to avoid FWER inflation.
7. Real-world example: In a study on the effect of different exercise programs on weight loss, the Bonferroni correction can be used to adjust the alpha level for multiple comparisons, and to avoid FWER inflation.
8. Misconception cleared: The Bonferroni correction is not a post hoc test, but rather a method used to adjust the alpha level for multiple comparisons.
9. Why is it important to use post hoc tests to avoid FWER inflation?
10. Answer: Post hoc tests are used to avoid FWER inflation, which occurs when multiple comparisons are made without adjusting for the increased risk of Type I errors.
11. Real-world example: In a study on the effect of different medications on blood pressure, post hoc tests can be used to avoid FWER inflation, and to ensure that the results are reliable.
12. Misconception cleared: FWER inflation is not a problem that occurs only in post hoc tests, but rather in any situation where multiple comparisons are made without adjusting for the increased risk of Type I errors.

HOW (process/application)

1. How is the Tukey's HSD test used to compare means of multiple groups?
2. Answer: The Tukey's HSD test is used to compare all possible pairs of means, and to determine which groups differ from each other.
3. Real-world example: In a study on the effect of different fertilizers on plant growth, the Tukey's HSD test can be used to compare the means of plant growth in each fertilizer group.
4. Misconception cleared: The Tukey's HSD test is not used to determine the overall significance of an ANOVA, but rather to identify which specific groups differ from each other.
5. How is the Bonferroni correction used to adjust the alpha level for multiple comparisons?
6. Answer: The Bonferroni correction is used to divide the alpha level by the number of comparisons, and to adjust the alpha level for multiple comparisons.
7. Real-world example: In a study on the effect of different exercise programs on weight loss, the Bonferroni correction can be used to adjust the alpha level for multiple comparisons.
8. Misconception cleared: The Bonferroni correction is not a post hoc test, but rather a method used to adjust the alpha level for multiple comparisons.
9. How are post hoc tests used to avoid FWER inflation?
10. Answer: Post hoc tests are used to avoid FWER inflation by comparing all possible pairs of means, and by adjusting the alpha level for multiple comparisons.
11. Real-world example: In a study on the effect of different medications on blood pressure, post hoc tests can be used to avoid FWER inflation, and to ensure that the results are reliable.
12. Misconception cleared: FWER inflation is not a problem that occurs only in post hoc tests, but rather in any situation where multiple comparisons are made without adjusting for the increased risk of Type I errors.

CAN (possibility/conditions)

1. Can the Tukey's HSD test be used to compare means of more than two groups?
2. Answer: Yes, the Tukey's HSD test can be used to compare means of more than two groups.
3. Real-world example: In a study on the effect of different fertilizers on plant growth, the Tukey's HSD test can be used to compare the means of plant growth in each fertilizer group.
4. Misconception cleared: The Tukey's HSD test is not limited to comparing means of only two groups.
5. Can the Bonferroni correction be used to adjust the alpha level for any type of multiple comparison?
6. Answer: No, the Bonferroni correction can only be used to adjust the alpha level for multiple comparisons that are independent.
7. Real-world example: In a study on the effect of different exercise programs on weight loss, the Bonferroni correction can be used to adjust the alpha level for multiple comparisons, but only if the comparisons are independent.
8. Misconception cleared: The Bonferroni correction is not suitable for all types of multiple comparisons.
9. Can post hoc tests be used to determine the overall significance of an ANOVA?
10. Answer: No, post hoc tests are used to identify which specific groups differ from each other, and not to determine the overall significance of an ANOVA.
11. Real-world example: In a study on the effect of different fertilizers on plant growth, post hoc tests can be used to compare the means of plant growth in each fertilizer group, but not to determine the overall significance of the ANOVA.
12. Misconception cleared: Post hoc tests are not used to determine the overall significance of an ANOVA, but rather to identify which specific groups differ from each other.

TRUE/FALSE (misconception testing)

1. Post hoc tests are used to determine the overall significance of an ANOVA.
2. Answer: FALSE
3. Real-world example: Post hoc tests are used to identify which specific groups differ from each other, and not to determine the overall significance of an ANOVA.
4. Misconception cleared: Post hoc tests are not used to determine the overall significance of an ANOVA, but rather to identify which specific groups differ from each other.
5. The Bonferroni correction is a post hoc test.
6. Answer: FALSE
7. Real-world example: The Bonferroni correction is a method used to adjust the alpha level for multiple comparisons, and not a post hoc test.
8. Misconception cleared: The Bonferroni correction is not a post hoc test, but rather a method used to adjust the alpha level for multiple comparisons.
9. Post hoc tests can be used to compare means of only two groups.
10. Answer: FALSE
11. Real-world example: Post hoc tests, such as the Tukey's HSD test, can be used to compare means of more than two groups.
12. Misconception cleared: Post hoc tests are not limited to comparing means of only two groups.