AI & Digital Ethics Grade 10: AI in Healthcare and Education

Grade 10 | AI & Digital Ethics Topic: AI in Healthcare and Education

1. The Driving Question

If a robot can diagnose your illness faster than a doctor or grade your essay in seconds, should we trust it more than a human—even if we don’t fully understand how it makes those decisions? And if an AI gets it wrong, who’s responsible: the programmer, the hospital, or the machine itself?

2. The Core Idea — Built, Not Listed

Imagine you’re in the emergency room with a high fever, and the doctor uses an AI tool called DeepMind Health to analyze your symptoms. The AI cross-references your data with millions of other cases and suggests a rare tropical disease—but the doctor disagrees and sends you home with antibiotics. A week later, you’re back, sicker, and the AI was right. Now what? This isn’t sci-fi; it’s happening in hospitals like Mount Sinai in New York, where AI predicts sepsis (a deadly infection) hours before doctors notice it.

In schools, picture Gradescope, an AI that grades handwritten math problems. It’s faster than a teacher, but what if it marks a student’s creative solution wrong because it doesn’t match the "expected" answer? Or worse, what if it unfairly penalizes students whose handwriting it struggles to read? These tools aren’t just tools—they’re decision-makers, and their choices have real consequences.

At the heart of this is a trade-off: AI can process more data, faster, than any human, but it lacks human judgment, empathy, and accountability. The question isn’t just can AI do these jobs—it’s should it, and how do we make sure it does them fairly?

Key Vocabulary: - Algorithmic Bias Definition: When an AI system produces unfair outcomes for certain groups because of flaws in its training data or design. Example: A facial-recognition AI used in schools to track attendance works poorly for students with darker skin tones, leading to higher false "absent" rates for Black students. College Note: In advanced ethics courses, bias is studied as a systemic issue—rooted in historical data, not just a "bug" to fix.

• Explainability Definition: The ability to understand and describe how an AI makes its decisions in a way humans can follow. Example: If an AI college-admissions tool rejects an applicant, the school should be able to explain why—not just say, "The AI said no." College Note: In fields like law, "black box" AI (where decisions can’t be explained) is increasingly challenged as a violation of due process.

• Autonomy (in AI Ethics) Definition: The degree to which humans retain control over decisions, even when AI is involved. Example: A hospital might use AI to suggest treatments, but doctors must have the final say—otherwise, patients lose their right to informed consent. College Note: Philosophers debate whether true AI autonomy (machines making independent choices) is even possible—or desirable.

• Data Privacy (in AI Contexts) Definition: The protection of personal information used to train or operate AI systems, especially when that data is sensitive (like medical records). Example: If a school uses an AI tutor that tracks students’ eye movements to detect confusion, who owns that data—the student, the school, or the tech company? College Note: Privacy laws (like GDPR in Europe) are evolving to address AI-specific risks, such as "re-identification" (piecing together anonymous data to reveal identities).

3. Assessment Translation

How This Appears on Assessments: - Classroom Debates/Short Essays: "Should AI be allowed to make final decisions in healthcare? Support your answer with at least two ethical principles." - State Standardized Tests (e.g., SBAC, PARCC): Multiple-choice questions on bias or privacy, often with scenarios like: "An AI hiring tool favors applicants from certain universities. This is an example of: (A) Efficiency (B) Algorithmic bias (C) Explainability (D) Autonomy." Distractor Pattern: Option (A) tricks students who focus on speed over fairness. - Research Projects: Analyze a real AI tool (e.g., IBM Watson for Oncology) and evaluate its strengths/weaknesses using ethical frameworks like utilitarianism (greatest good for the greatest number) or deontology (duty-based rules).

Proficient vs. Developing Responses: | Proficient | Developing | |----------------|----------------| | Prompt: "Explain one risk of using AI in education and how schools could address it." | | | Response: "One risk is that AI grading tools might penalize students for creative answers that don’t match the AI’s training data. For example, if a student solves a math problem in an unusual way, the AI might mark it wrong even if it’s correct. Schools could address this by having teachers review a sample of AI-graded work to check for errors and by training the AI on more diverse examples." | Response: "AI in schools is bad because it’s not fair. It might make mistakes." (Lacks specific example, solution, or ethical reasoning.) | | What the Teacher Looks For: Specific scenario, named tool, ethical concern, and a practical solution. | What’s Missing: Vague language, no connection to bias/explainability, no actionable fix. |

Model Proficient Response (Short Essay): "AI in healthcare, like Google’s DeepMind, can detect diseases faster than doctors, but it also risks reinforcing biases. For example, if the AI is trained mostly on data from white patients, it might misdiagnose conditions in people of color. This violates the ethical principle of justice—treating all patients fairly. Hospitals could address this by diversifying training data and requiring human doctors to double-check AI recommendations, especially for underrepresented groups. However, this slows down the process, creating a trade-off between speed and fairness. The key is transparency: patients should know when AI is used and how it might affect their care."

4. Mistake Taxonomy

Mistake 1: Overgeneralizing AI’s Capabilities - Prompt: "Describe one benefit of using AI in education." - Common Wrong Response: "AI is better than teachers because it’s smarter and never makes mistakes." - Why It Loses Credit: Ignores AI’s limitations (bias, lack of empathy) and oversimplifies human roles. - Correct Approach: "AI can personalize learning by adapting to students’ pace, like Khan Academy’s math tutor. However, it can’t replace teachers’ ability to motivate students or address emotional needs. A balanced approach uses AI for repetitive tasks (like grading) while keeping teachers for mentorship."

Mistake 2: Confusing Correlation with Causation in Bias - Prompt: "An AI college-admissions tool rejects more female applicants for STEM programs. Is this bias? Explain." - Common Wrong Response: "No, it’s just reflecting the data—fewer women apply to STEM." - Why It Loses Credit: Fails to recognize that the AI might amplify existing biases (e.g., if past admissions data favored men, the AI learns to favor men). - Correct Approach: "This could be bias if the AI’s training data included historical discrimination. For example, if past admissions committees favored male applicants, the AI might learn to associate ‘male’ with ‘good STEM candidate.’ Schools should audit the AI’s decisions and retrain it with fairer data."

Mistake 3: Ignoring Stakeholder Perspectives - Prompt: "Should hospitals use AI to prioritize patients in the ER? Give one argument for and one against." - Common Wrong Response: "Yes, because it’s faster. No, because it might be wrong." (Too vague; no ethical reasoning.) - Why It Loses Credit: Doesn’t consider specific stakeholders (patients, doctors, insurers) or ethical principles. - Correct Approach: "For: AI could reduce wait times by quickly identifying critical cases, like sepsis, which saves lives. Against: Patients might distrust AI decisions, especially if they’re from marginalized groups that AI has misdiagnosed in the past. Hospitals must balance efficiency with autonomy—patients should have the right to a human second opinion."

5. Connection Layer

1. Within AI & Digital Ethics-AI in Criminal Justice Why it matters: Both healthcare and criminal justice AI (e.g., COMPAS for sentencing) rely on predictive algorithms, but errors in either can ruin lives. Understanding bias in one helps you spot it in the other.

2. Across Subjects-Biology (Genetics) & AI Why it matters: AI analyzes genetic data to predict diseases, but this raises privacy questions (e.g., could insurers deny coverage based on AI-predicted risks?). The same ethical dilemmas appear in both fields: Who owns your genetic data?

3. Outside School-Social Media Algorithms Why it matters: The same algorithmic bias that misdiagnoses patients can amplify misinformation on platforms like TikTok. If an AI prioritizes engagement over truth, it might push harmful health advice (e.g., anti-vaccine content) to vulnerable users.

6. The Stretch Question

If an AI therapist (like Woebot) gives you mental health advice, and you follow it but get worse, who’s responsible—the AI, the company that made it, or you?

Pointer Toward the Answer: This isn’t just about blame—it’s about accountability gaps. In healthcare, doctors are legally liable for malpractice, but AI companies often hide behind terms like "not a medical device" to avoid responsibility. Some ethicists argue we need new laws for "AI malpractice," while others say humans should never delegate life-or-death decisions to machines. The bigger question: Can we design AI that’s both autonomous and accountable, or is that a contradiction?

Tone Note: This guide assumes Grade 10 students are ready to grapple with nuance—no hand-holding, but no jargon without context. The examples are real (DeepMind, Gradescope), the stakes are clear (life, fairness, privacy), and the connections show how this topic ripples into other areas of their lives.