K-12 Math (US): 9-12 Data Analysis K-12 Math Statistics Sampling

Study Guide: Statistics — Sampling (Grades 9–12, Math)

1. The Driving Question

"If you want to know how many high schoolers in your state support a new school lunch policy, how can you figure it out without asking every single student? And why does it matter who you ask—or how you ask them?" This isn’t just about saving time—it’s about whether your answer is true or just a lucky guess. If you ask 100 students at a football game, is that the same as asking 100 students in the library? How do you know when your sample is telling you something real about the whole population, and when it’s just noise?

2. The Core Idea — Built, Not Listed

Imagine you’re a marine biologist studying a school of 10,000 fish in a lake. You can’t count every fish, so you net 200 at random spots. If 15% of those fish have a rare parasite, you might guess that 15% of the whole school does too. But what if you only net fish near the shore, where the water is warmer? Your sample might overcount infected fish, because the parasite thrives in heat. That’s the puzzle of sampling: your answer is only as good as how you pick your sample.

A good sample is like a well-mixed smoothie—every sip should taste the same, because every part of the blender got blended. If you only scoop from the top, you might get all strawberry and no banana. In statistics, that’s called bias: when your sample doesn’t represent the whole population, and your answer gets skewed.

Here’s how it works: - Population: The entire group you care about (e.g., all high schoolers in your state).
- Sample: The smaller group you actually measure (e.g., 500 students from 10 schools).
- Random sampling: Every member of the population has an equal chance of being picked (like drawing names from a hat).
- Bias: When your sample is systematically different from the population (e.g., only surveying students in the cafeteria at lunch).

Key Vocabulary:
- Population: The entire group you want to study.
Example: All registered voters in Ohio, not just the ones who show up to a rally.
College shift: In advanced stats, populations can be infinite (e.g., "all possible coin flips"), and sampling becomes about probability distributions.

• Sample: A subset of the population used to make inferences.
  Example: Testing 100 lightbulbs from a factory to estimate the failure rate of 10,000.
  College shift: Samples are often analyzed using distributions (e.g., t-distribution for small samples), not just point estimates.

• Bias: A systematic error that makes your sample unrepresentative.
  Example: A survey about homework stress only given to students in AP classes (they’re likely more stressed than average).
  College shift: Bias is quantified in terms of expected value (e.g., "the estimator is biased if its expected value doesn’t equal the population parameter").

• Margin of Error: How much your sample result might differ from the true population value.
  Example: A poll says 52% of voters support a candidate, with a ±3% margin of error. The true support is likely between 49% and 55%.
  College shift: Margin of error is derived from the standard error of the sampling distribution, which depends on sample size and variability.

3. Assessment Translation

How this appears on assessments:
- SAT/ACT: Multiple-choice questions about identifying bias in sampling methods or interpreting margins of error. Example: "A researcher surveys 200 people at a mall on a Tuesday morning to estimate the average number of hours Americans spend on social media. Which of the following is the most likely source of bias in this study?" Distractors often include: - Confusing sample size with bias (e.g., "The sample is too small" — but size affects precision, not bias).
- Misidentifying the population (e.g., "The population is mall shoppers" — no, the population is "Americans").
- Overlooking time/location effects (e.g., "The survey was only on Tuesday" — but this might not matter if the question is about general habits).

• AP Statistics: Free-response questions (FRQs) where you design a sampling method, identify flaws, or calculate margins of error. Example: "A school newspaper wants to estimate the proportion of students who support a new dress code. They survey 50 students from the senior class. Identify two sources of bias in this sampling method and explain how each could affect the estimate." Rubric priorities:
• Score of 4: Clearly identifies bias (e.g., "seniors may have different opinions than underclassmen") and explains the direction of the effect (e.g., "this could overestimate support if seniors are more likely to oppose the dress code").
• Score of 2: Identifies bias but doesn’t explain the effect, or explains vaguely (e.g., "this is bad because it’s not random").

Model Proficient Response (AP FRQ):
Prompt: "A city council wants to estimate the average number of hours per week residents spend volunteering. They survey 100 people at a food bank. Identify one source of bias and explain how it could affect the estimate." Response: "One source of bias is volunteer bias: the sample only includes people who are already volunteering at a food bank, so they likely spend more time volunteering than the average resident. This would cause the estimate to be too high, because the sample isn’t representative of the whole population."

4. Mistake Taxonomy

Mistake 1: Confusing Sample Size with Representativeness
Question: "A researcher surveys 1,000 people at a political rally to estimate the candidate’s support among all voters. Is this a good sample? Why or why not?" Common Wrong Response: "Yes, because 1,000 is a large sample size." Why It Loses Credit: The question is about bias, not precision. A large sample size reduces margin of error but doesn’t fix bias. The response ignores that the sample is only rally attendees, who are likely more supportive than the general population.
Correct Approach: 1. Identify the population: all voters.
2. Identify the sample: rally attendees.
3. Explain the bias: rally attendees are not representative of all voters (they’re likely more supportive).
4. Conclude: The sample is biased, so the estimate will be too high.

Mistake 2: Misidentifying the Population
Question: "A study surveys 500 high school students to estimate the average number of hours teens spend on homework. What is the population?" Common Wrong Response: "The population is the 500 students surveyed." Why It Loses Credit: The population is the group you’re making inferences about, not the sample. The response confuses the sample with the population.
Correct Approach: 1. The population is all teens (or "all high school students," depending on context).
2. The sample is the 500 students surveyed.
3. The goal is to use the sample to estimate a parameter (e.g., mean homework hours) for the population.

Mistake 3: Ignoring Nonresponse Bias
Question: "A mail survey is sent to 1,000 households to estimate the percentage of people who recycle. Only 300 respond. What is one potential source of bias?" Common Wrong Response: "The sample size is too small." Why It Loses Credit: The issue isn’t sample size—it’s that the 700 who didn’t respond might differ systematically from those who did (e.g., people who recycle might be more likely to respond). This is nonresponse bias.
Correct Approach: 1. Identify the bias: nonresponse bias.
2. Explain the effect: If non-recyclers are less likely to respond, the estimate will be too high.
3. Suggest a fix: Follow up with nonresponders or compare demographics of responders vs. nonresponders.

5. Connection Layer

1. Within Math: Sampling → Probability Distributions
   Why it matters: The margin of error in a sample comes from the sampling distribution of the statistic (e.g., the mean). Understanding sampling helps you grasp why the Central Limit Theorem works—it’s why large samples give normal distributions, even if the population isn’t normal.

2. Across Subjects: Sampling → Scientific Method (Science)
   Why it matters: In science, experiments are like samples of nature. A drug trial with 100 people is a sample of all possible patients. If the sample isn’t random (e.g., only young, healthy people), the results won’t generalize—just like a biased survey.

3. Outside School: Sampling → A/B Testing (Tech/Marketing)
   Why it matters: Companies like Netflix or Amazon use sampling to test changes. If they show a new layout to 1% of users, that’s a sample. If they only test it on desktop users, the sample is biased (mobile users might react differently). Now you’ll notice when a website looks "off" because they didn’t sample well.

6. The Stretch Question

"If you survey 1,000 people and find that 60% support a policy, with a ±3% margin of error, does that mean the true support is definitely between 57% and 63%? Why or why not?"

Pointer Toward the Answer: The margin of error is a probability statement, not a guarantee. It means that if you repeated the survey many times, 95% of the intervals you calculate (e.g., 57%–63%) would contain the true population value. But there’s still a 5% chance the true value is outside that range. It’s like saying, "I’m 95% sure the treasure is in this room"—but the treasure could still be in the hallway. The margin of error assumes your sample is unbiased; if it’s not, the interval might not even be close.