Statistics 101
Random

Click random to get a fresh chapter.

Introductory Statistics: Advanced Topics - Statistical vs. Practical Significance Effect Sizes Cohens d r

What Is This?

Statistical vs. Practical Significance refers to the distinction between whether a result is statistically significant (i.e., unlikely to have occurred by chance) and whether it is practically meaningful or important in real-world terms. Effect sizes (like Cohen's d and r) measure the magnitude of a difference or relationship, helping to determine practical significance. This topic appears in exams to test your ability to interpret statistical results in a meaningful context.

Why It Matters

This topic is tested in statistics, psychology, and research methods exams. It frequently appears in questions worth 10-15% of the total marks. It tests your ability to critically evaluate research findings and understand the real-world implications of statistical results.

Core Concepts

1. Statistical Significance: A result is statistically significant if it is unlikely to have occurred by chance. This is typically determined using p-values.
2. Practical Significance: A result is practically significant if it has real-world importance or meaningful impact.
3. Effect Sizes: Measures like Cohen's d and r quantify the magnitude of differences or relationships.
4. Cohen's d: Measures the difference between two means in standard deviation units.
5. r (Correlation Coefficient): Measures the strength and direction of a linear relationship between two variables.

Prerequisites

1. Understanding of p-values: Knowing how p-values determine statistical significance.
2. Basic Descriptive Statistics: Knowing means, standard deviations, and correlation concepts.
3. Research Methods: Understanding the context of hypothesis testing and experimental design.

The Rule-Book (How It Works)

Primary Rule

Statistical significance does not imply practical significance. Always consider effect sizes to determine the real-world impact.

Sub-rules and Exceptions

• Cohen's d: Use when comparing two means. A d of 0.2 is small, 0.5 is medium, and 0.8 is large.
• r (Correlation Coefficient): Use for linear relationships. An r of 0.1 is small, 0.3 is medium, and 0.5 is large.
• Edge Cases: Very large sample sizes can produce statistically significant results with tiny effect sizes, emphasizing the need to check practical significance.

Visual Pattern

Think of effect sizes as the "volume knob" on your results: statistical significance tells you there's a signal, but effect sizes tell you how loud it is.

Exam / Job / Audit Weighting

• Frequency: Common
• Difficulty Rating: Intermediate
• Question Type: Multiple choice, short answer, data interpretation

Difficulty Level

Intermediate

Must-Know Rules, Formulas, Standards, or Principles

1. Cohen's d Formula: ( d = \frac{M_1 - M_2}{SD_{pooled}} )
2. Interpreting Cohen's d:
3. Small: 0.2
4. Medium: 0.5
5. Large: 0.8
6. Interpreting r:
7. Small: 0.1
8. Medium: 0.3
9. Large: 0.5

Worked Examples (Step-by-Step)

Easy

Question: A study finds a statistically significant difference (p < 0.05) between two groups with a Cohen's d of 0.1. Is this result practically significant? Reasoning:
1. Identify Cohen's d value: 0.1.
2. Compare to benchmarks: 0.1 is considered a small effect size.
3. Conclusion: The result is statistically significant but not practically significant. Answer: No, the result is not practically significant.

Medium

Question: Two groups have means of 50 and 55 with a pooled standard deviation of 10. Calculate Cohen's d and interpret its practical significance. Reasoning:
1. Use Cohen's d formula: ( d = \frac{55 - 50}{10} = 0.5 ).
2. Compare to benchmarks: 0.5 is a medium effect size.
3. Conclusion: The result has medium practical significance. Answer: Cohen's d is 0.5, indicating medium practical significance.

Hard

Question: A correlation study finds r = 0.4 between two variables with a p-value of 0.01. Interpret the statistical and practical significance. Reasoning:
1. Identify r value: 0.4.
2. Compare to benchmarks: 0.4 is between medium and large effect size.
3. p-value indicates statistical significance.
4. Conclusion: The result is statistically significant and has a medium to large practical significance. Answer: The result is statistically significant with a medium to large practical significance.

Common Exam Traps & Mistakes

1. Mistake: Assuming statistical significance means practical significance.
2. Wrong Answer: A p-value of 0.01 means the result is important.
3. Correct Approach: Check the effect size to determine practical significance.
4. Mistake: Misinterpreting Cohen's d benchmarks.
5. Wrong Answer: A Cohen's d of 0.3 is large.
6. Correct Approach: 0.3 is a small to medium effect size.
7. Mistake: Ignoring sample size impact on statistical significance.
8. Wrong Answer: A large sample size always means the result is important.
9. Correct Approach: Large samples can produce significant results with small effect sizes.

Shortcut Strategies & Exam Hacks

• Memory Aid: Remember "0.2, 0.5, 0.8" for Cohen's d and "0.1, 0.3, 0.5" for r.
• Elimination Strategy: If a question asks about practical significance, eliminate options that only discuss p-values.
• Pattern Recognition: Look for keywords like "significant," "effect size," and "practical importance" to focus your approach.

Question-Type Taxonomy

1. Multiple Choice: Common in standardized tests.
2. Example: What does a Cohen's d of 0.6 indicate?
   • A) Small effect size
   • B) Medium effect size
   • C) Large effect size
   • D) No effect size
3. Short Answer: Common in course exams.
4. Example: Explain the difference between statistical and practical significance.
5. Data Interpretation: Common in research methods exams.
6. Example: Interpret the practical significance of a study with r = 0.25 and p < 0.05.

Practice Set (MCQs)

1. Question: A study finds a statistically significant difference (p < 0.05) with a Cohen's d of 0.3. Is this result practically significant?
2. Options:
   • A) Yes, it is large.
   • B) Yes, it is medium.
   • C) No, it is small.
   • D) Cannot determine.
3. Correct Answer: C) No, it is small.
4. Explanation: Cohen's d of 0.3 is a small effect size.

5. Why the Distractors Are Tempting: A and B suggest larger effect sizes, D implies uncertainty.

6. Question: What does an r value of 0.45 indicate?

7. Options:
   • A) Small effect size
   • B) Medium effect size
   • C) Large effect size
   • D) No effect size
8. Correct Answer: B) Medium effect size.
9. Explanation: r of 0.45 is between medium and large.

10. Why the Distractors Are Tempting: A and C suggest incorrect effect sizes, D implies no effect.

11. Question: A study with a large sample size finds a statistically significant result (p < 0.01) but a Cohen's d of 0.1. What can you conclude?

12. Options:
    • A) The result is practically significant.
    • B) The result is not practically significant.
    • C) The result is both statistically and practically significant.
    • D) The result is neither statistically nor practically significant.
13. Correct Answer: B) The result is not practically significant.
14. Explanation: Cohen's d of 0.1 is small, despite statistical significance.

15. Why the Distractors Are Tempting: A and C suggest practical significance, D denies statistical significance.

16. Question: Which of the following is true about statistical vs. practical significance?

17. Options:
    • A) Statistical significance always implies practical significance.
    • B) Practical significance always implies statistical significance.
    • C) Statistical significance does not imply practical significance.
    • D) Both are always independent of each other.
18. Correct Answer: C) Statistical significance does not imply practical significance.
19. Explanation: Statistical significance requires checking effect sizes for practical significance.

20. Why the Distractors Are Tempting: A and B suggest a direct relationship, D implies complete independence.

21. Question: A correlation study finds r = 0.2 with a p-value of 0.03. What is the practical significance?

22. Options:
    • A) Small
    • B) Medium
    • C) Large
    • D) None
23. Correct Answer: A) Small
24. Explanation: r of 0.2 is a small effect size.
25. Why the Distractors Are Tempting: B and C suggest larger effect sizes, D implies no effect.

30-Second Cheat Sheet

• Statistical significance-practical significance.
• Cohen's d: 0.2 (small), 0.5 (medium), 0.8 (large).
• r: 0.1 (small), 0.3 (medium), 0.5 (large).
• Always check effect sizes for practical significance.
• Large samples can produce significant results with small effect sizes.

Learning Path

1. Beginner Foundation: Understand p-values and basic descriptive statistics.
2. Core Rules: Learn Cohen's d and r formulas and interpretations.
3. Practice: Solve example problems and interpret research findings.
4. Timed Drills: Practice under exam conditions.
5. Mock Tests: Take full-length practice exams.

Related Topics

1. Hypothesis Testing: Understanding p-values and statistical significance.
2. Descriptive Statistics: Means, standard deviations, and correlation concepts.
3. Research Methods: Experimental design and data interpretation.