AI & Digital Ethics Grade 6 Bias in AI Why Algorithms Can Be Unfair

Grade 6 | AI & Digital Ethics
Topic: Bias in AI: Why Algorithms Can Be Unfair

1. The Driving Question

"If a robot or a computer program is supposed to be ‘smart,’ why does it sometimes make unfair decisions—like favoring one group of people over another? And how can something that doesn’t even have feelings still end up being biased?"

This isn’t just about robots being "mean." It’s about how the rules we teach machines can accidentally carry the same blind spots we have—and how fixing that might mean rethinking how we design technology in the first place.

2. The Core Idea — Built, Not Listed

Imagine you’re training a new student, Alex, to sort colored pencils into bins. You show Alex 100 pencils: 90 are blue, 5 are red, and 5 are green. Every time Alex sees a blue pencil, you say, "Great job!" But when Alex picks up a red or green one, you don’t say anything. After a week, Alex starts ignoring red and green pencils entirely—even when you want them sorted. Alex isn’t being stubborn; they’re just following the rules you set up.

AI works the same way. Algorithms (the step-by-step rules computers follow) learn from data—the examples we give them. If that data is unbalanced (like mostly blue pencils), the AI will favor the "blue pencils" of the world, even if that means overlooking other colors. Worse, if the data reflects real-world biases (like more photos of one race in a facial-recognition database), the AI will repeat those biases faster and at scale. The problem isn’t that the AI is "prejudiced"; it’s that it’s mirroring the patterns we’ve already created—without questioning them.

Key Vocabulary:
- Algorithm: A set of step-by-step instructions a computer follows to solve a problem or make a decision.
Example: The recipe a navigation app uses to pick the fastest route—it doesn’t "know" traffic; it just follows rules like "avoid highways if they’re congested." Note for later: In college, algorithms are studied for their efficiency (how fast they work) and their fairness (who gets left out).

• Bias (in AI): When an algorithm produces unfair or inaccurate results for certain groups because of flaws in its training data or design.
  Example: A hiring AI that rejects resumes with the word "women’s" (like "women’s chess club") because it was trained on past hiring data where most hires were men.
  Note for later: Bias isn’t always intentional—it can be statistical (like the pencil example) or historical (like data reflecting past discrimination).

• Training Data: The examples (images, text, numbers) used to teach an AI how to make decisions.
  Example: A voice assistant trained on recordings of 10,000 people—but 9,000 of them speak with a Southern U.S. accent. It might struggle to understand other accents.
  Note for later: In advanced AI, training data is scrutinized for representativeness (does it cover all groups?) and labeling (are the examples tagged fairly?).

• Feedback Loop: When an AI’s biased output is fed back into its training, making the bias worse over time.
  Example: A social media algorithm that shows more police arrest videos to Black users because past users engaged with them—then uses that engagement to justify showing more arrest videos, reinforcing stereotypes.
  Note for later: Feedback loops are a major focus in AI ethics and critical data studies.

3. Assessment Translation

How This Appears on State Assessments (Grade 6):
- Multiple Choice: Questions test understanding of how bias enters AI (e.g., "Which of these is the most likely reason a facial-recognition AI fails to recognize darker-skinned faces?").
Distractor patterns: - Confusing intentional bias (e.g., "The programmers were racist") with unintentional bias (e.g., "The training data was unbalanced").
- Assuming AI "thinks" like a human (e.g., "The AI chose to ignore certain groups").
- Short Answer: Prompts like "Explain one way an AI’s training data could lead to biased results. Use an example." - Proficient response: Names a specific type of data (e.g., "photos of CEOs") and explains how imbalance (e.g., "mostly white men") leads to unfair outcomes (e.g., "the AI might think only white men can be leaders").
- Developing response: Vague (e.g., "the data was bad") or misidentifies the cause (e.g., "the AI was programmed to be racist").
- Evidence-Based Writing: "Some people say AI is ‘neutral’ because it’s just math. Do you agree? Use evidence from the text and your own reasoning." - Proficient response: Acknowledges math is neutral but argues data and design choices aren’t (e.g., "Math doesn’t have opinions, but the people who pick the training data do").

Model Proficient Response (Short Answer):
"An AI used to screen job applications might be biased if its training data only includes resumes from people who went to expensive colleges. For example, if 90% of the resumes it learned from were from Ivy League schools, the AI might start thinking only Ivy League graduates are qualified—even if great candidates went to other schools. This isn’t because the AI is ‘snobby’; it’s because it was only shown one type of example."

4. Mistake Taxonomy

Mistake 1: Blaming the AI for "having opinions"
- Prompt: "Why might a loan-approval AI reject applications from people in certain neighborhoods?" - Common wrong response: "The AI is racist because it doesn’t like those neighborhoods." - Why it loses credit: Attributes human-like intent to the AI. Assessments want explanations rooted in data or design, not anthropomorphism.
- Correct approach: 1. Identify the training data (e.g., "past loan approvals from that bank").
2. Note the imbalance (e.g., "most approved loans were in wealthy areas").
3. Explain the outcome (e.g., "the AI learns to associate ‘good loans’ with those areas").
4. Flag the real-world consequence (e.g., "people in poorer neighborhoods get rejected even if they’re qualified").

Mistake 2: Confusing correlation with causation
- Prompt: "A study found that an AI used in hospitals recommended fewer pain treatments for Black patients. What’s one possible reason for this?" - Common wrong response: "Black patients don’t feel as much pain." - Why it loses credit: Misinterprets the AI’s output as proof of a biological difference, rather than a flaw in the data. This is a classic correlation ≠ causation error.
- Correct approach: - The AI’s recommendation is based on past data (e.g., "doctors prescribed fewer pain meds to Black patients").
- The data reflects human bias, not medical truth.
- The AI amplifies this bias by assuming past patterns = correct patterns.

Mistake 3: Suggesting "more data" is always the fix
- Prompt: "How could you reduce bias in an AI that predicts which students will drop out of school?" - Common wrong response: "Add more data about all the students." - Why it loses credit: Assumes quantity fixes quality. More data can worsen bias if it’s still unbalanced (e.g., adding 1,000 more records from wealthy schools doesn’t help if the AI needs examples from underfunded schools).
- Correct approach: - Audit the existing data for gaps (e.g., "Are rural schools represented?").
- Collect targeted data to fill those gaps (e.g., "partner with schools in different regions").
- Test the AI’s decisions for fairness (e.g., "Does it flag boys more than girls for the same behavior?").

5. Connection Layer

• Within AI & Digital Ethics → Deepfakes and Misinformation: Bias in AI isn’t just about unfair decisions—it’s also about who gets to control the narrative. A deepfake AI trained mostly on videos of white men will struggle to generate realistic images of women or people of color, which can shape whose voices are amplified (or faked) online.

• Across Subjects → History (Redlining) → Math (Sampling Bias): The practice of redlining (denying loans to neighborhoods based on race) created maps that banks still use today—including in AI training data. This is a real-world example of sampling bias in math, where a non-representative sample (e.g., "only wealthy neighborhoods") leads to skewed conclusions.

• Outside School → TikTok’s "For You" Page: The algorithm that decides what videos you see is trained on your past behavior—and the behavior of people like you. If you watch a lot of cooking videos, you’ll see more cooking videos. But if the AI assumes "people like you" only like certain things (e.g., "girls like makeup tutorials"), it can limit what you discover. This is algorithmic bias in action—and it’s why you might see the same types of creators over and over.

6. The Stretch Question

"If an AI is biased because of its training data, is it ever okay to use biased data to train an AI? For example, could a biased dataset be the ‘lesser evil’ in some cases?"

Pointer Toward the Answer:
This isn’t a yes/no question—it’s about trade-offs. For example: - A hospital AI trained on biased historical data might still save lives by flagging overlooked patterns (e.g., "this symptom is often missed in women").
- But using that same data to predict future health risks could reinforce the bias (e.g., "women’s pain is taken less seriously").
The real question is: Who gets to decide when the benefits outweigh the harm? And how do we make sure those decisions are transparent? (Hint: This is why AI ethicists talk about accountability and audits.)