AP Statistics (AP Stats): Bias in Sampling (Voluntary Response, Undercoverage, Non?response)

AP Statistics – Bias in Sampling (Voluntary Response, Undercoverage, Non?response)

AP Statistics Study Guide: Bias in Sampling (Voluntary Response, Undercoverage, Non-response)

What This Is

Bias in sampling occurs when a sample systematically favors certain outcomes, leading to inaccurate estimates of a population parameter. On the AP exam, you’ll need to identify, explain, and suggest fixes for different types of bias in survey designs. For example, if a radio show asks listeners to call in and vote on a political issue, the results may not reflect the true opinions of the entire population due to voluntary response bias. Understanding bias is crucial for designing valid studies and interpreting survey results correctly.

Key Terms & Formulas

• Bias: Systematic error that leads to inaccurate estimates of a population parameter.
• Voluntary Response Bias: Occurs when individuals self-select into a sample (e.g., call-in polls, online surveys). Those with strong opinions are overrepresented.
• Undercoverage Bias: Happens when some groups in the population are left out of the sampling frame (e.g., a phone survey misses people without landlines).
• Non-response Bias: Occurs when individuals chosen for a sample do not respond, and their responses may differ from those who do (e.g., a mailed survey with a 30% response rate).
• Sampling Frame: The list of individuals from which a sample is drawn (e.g., a school’s student directory).
• Random Sampling: A method to reduce bias by giving every individual in the population an equal chance of being selected (e.g., simple random sample, stratified sample).
• Stratified Random Sample: Divides the population into subgroups (strata) and takes a random sample from each (e.g., sampling 50 students from each grade level).
• Cluster Sample: Divides the population into clusters, randomly selects some clusters, and samples all individuals within them (e.g., randomly selecting 5 homerooms and surveying all students in them).
• Systematic Sample: Selects every k-th individual from a list (e.g., surveying every 10th person in a phone book).
• Response Bias: Occurs when respondents answer questions inaccurately (e.g., due to leading questions, social desirability, or interviewer influence).

Step-by-Step / Process Flow

How to Analyze Bias in an FRQ

1. Identify the Sampling Method
2. Is it a voluntary response, convenience sample, random sample, etc.?

3. Example: "A radio host asks listeners to call in and vote on a new policy"-voluntary response.

4. Determine the Type of Bias

5. Ask: Who is overrepresented or underrepresented?

6. Example: "Only passionate listeners call in"-voluntary response bias.

7. Explain the Impact on Results

8. How does the bias affect the estimate? (Overestimate? Underestimate?)

9. Example: "The survey will likely overestimate support for the policy because only people with strong opinions will respond."

10. Suggest a Better Sampling Method

11. Propose a random sampling technique (e.g., simple random sample, stratified sample).

12. Example: "Instead, use a simple random sample of registered voters to ensure every voter has an equal chance of being selected."

13. Check for Other Biases

14. Could non-response or undercoverage also be an issue?
15. Example: "If the survey is mailed, non-response bias could occur if only 20% of people respond."

Common Mistakes

• Mistake: Confusing voluntary response bias with non-response bias.

• Correction: Voluntary response = people choose to participate (e.g., online polls). Non-response = people are selected but don’t respond (e.g., mail surveys).

• Mistake: Saying a convenience sample (e.g., surveying friends) is unbiased.

• Correction: Convenience samples are not random and often lead to undercoverage bias.

• Mistake: Ignoring undercoverage when the sampling frame is incomplete.

• Correction: Always check if the sampling frame excludes certain groups (e.g., a phone survey misses people without phones).

• Mistake: Assuming larger samples eliminate bias.

• Correction: A large sample with bias is still biased! (e.g., a million people calling into a radio show still overrepresent strong opinions.)

• Mistake: Forgetting to explain the direction of bias (over/underestimate).

• Correction: Always state whether the bias leads to an overestimate or underestimate of the true parameter.

AP Exam Insights

• FRQs often ask you to:
• Identify a type of bias in a given scenario.
• Explain how the bias affects the results (e.g., "The survey will overestimate support for the policy").
• Suggest a better sampling method to reduce bias.
• Common setups:
• A school newspaper survey (voluntary response).
• A phone survey (undercoverage of people without phones).
• A mailed survey (non-response bias).
• Tricky distinction: Non-response bias vs. undercoverage—non-response is about selected people not responding, while undercoverage is about missing groups entirely.
• Calculator tip: While bias isn’t directly calculated, you may need to simulate random sampling (e.g., randInt(1, N, n) for a simple random sample).

Quick Check Questions

1. Multiple Choice

A city council wants to estimate the proportion of residents who support a new park. They post a survey on the city’s website and receive 500 responses. What type of bias is most likely present? (A) Undercoverage bias (B) Non-response bias (C) Voluntary response bias (D) Response bias

Answer: (C) Voluntary response bias Explanation: Only people who visit the website and choose to respond are included, overrepresenting those with strong opinions.

2. FRQ (Short Answer)

A researcher wants to estimate the average number of hours high school students spend on homework per night. They randomly select 100 students from a single school and ask them to report their homework time. (a) Identify a potential source of bias in this study. (b) Explain how this bias could affect the estimate of the average homework time.

Answer: (a) Undercoverage bias (if the school is not representative of all high schools) or response bias (if students over/underreport homework time). (b) If the school has unusually high/low homework loads, the estimate will be too high/low. If students overreport, the estimate will be inflated.

Last-Minute Cram Sheet

1. Voluntary response bias = people choose to participate (e.g., call-in polls).
2. Undercoverage bias = some groups are left out of the sampling frame (e.g., phone surveys miss people without phones).
3. Non-response bias = selected individuals don’t respond (e.g., mail surveys with low response rates).
4. Response bias = answers are inaccurate (e.g., leading questions, social desirability).
5. Random sampling reduces bias (e.g., SRS, stratified, cluster).
6. Larger samples don’t fix bias! (A biased sample of 1 million is still biased.)
7. Always explain the direction of bias (over/underestimate).
8. Convenience samples are biased (e.g., surveying friends).
9. Check for multiple biases (e.g., a phone survey could have undercoverage and non-response bias).
10. For FRQs, suggest a better method (e.g., "Use a simple random sample of all students").