AI & Digital Ethics Grade 10 Large Language Models How GPT-style AI Works

Study Guide: Large Language Models – How GPT-Style AI Works
Grade 10 | AI & Digital Ethics

1. The Driving Question

"If I ask an AI like ChatGPT to write a poem about a robot falling in love with a toaster, how does it come up with something that sounds like a human wrote it—without ever having seen a toaster or felt love? And why does it sometimes get weirdly wrong, like insisting a toaster has a soul?" This isn’t just about how AI "thinks"—it’s about how a machine can seem to understand language when it’s really just predicting the next word in a sentence, over and over. By the end, you’ll know why these systems are both brilliant and brittle, and why that matters for everything from homework to deepfakes.

2. The Core Idea – Built, Not Listed

Imagine you’re playing the world’s most chaotic game of Mad Libs, but instead of filling in blanks with nouns and verbs, you’re trying to guess the next word in a sentence based on the last 500 words you’ve read. That’s essentially what a large language model (LLM) like GPT does—but it’s trained on trillions of words from books, websites, and conversations, so it’s really good at spotting patterns.

Here’s the scenario: You type "The cat sat on the ___" into an LLM. The AI doesn’t "know" what a cat is, but it’s seen this phrase (or similar ones) millions of times in its training data. It calculates probabilities: "mat" (30%), "couch" (25%), "roof" (10%), "moon" (0.001%). It picks the most likely word, then moves to the next blank: "and purred ___." Again, it guesses based on patterns: "softly" (40%), "loudly" (15%), "at a squirrel" (5%). It doesn’t understand the sentence—it’s just stringing together statistically probable words, like a hyper-advanced autocomplete.

This process is called next-token prediction (where a "token" is a word or part of a word). The "large" in LLM comes from two things: 1. Massive training data: GPT-4 was trained on text equivalent to millions of books—far more than any human could read in a lifetime.
2. Neural networks: These are layers of mathematical "neurons" that adjust their connections (like tuning a radio) to get better at predicting the next word. The more layers (or "parameters"), the more nuanced the predictions—but also the more computing power required.

The result? An AI that can write essays, debug code, or even mimic Shakespeare… but also one that can confidently hallucinate fake facts, repeat biases from its training data, or fail spectacularly when asked about something outside its "experience" (like that toaster with a soul).

Key Vocabulary:
- Token
- Definition: The smallest unit of text an LLM processes—usually a word, but sometimes a subword (like "un-" or "-ing") or even a single character.
- Example: In the sentence "AI is cool!", the tokens might be ["AI", "is", "cool", "!"]. In "Unbelievable!", it could be ["Un", "believ", "able", "!"]. (This is why LLMs sometimes struggle with rare words or typos—they’re breaking them into unfamiliar pieces.) - College note: In computational linguistics, tokenization gets way more complex with languages like Chinese (no spaces between words) or code (where ";" and "{" are critical tokens). Some models now use "byte-pair encoding" to handle this.

• Neural Network (Transformer Architecture)
• Definition: A system of layered mathematical functions that processes input data (like text) and learns patterns by adjusting the strength of connections between "neurons" (tiny calculators).
• Example: Think of a neural network like a Rube Goldberg machine where each part (neuron) tweaks the input slightly before passing it to the next. In a transformer, the "attention mechanism" (a key part) acts like a spotlight, focusing on the most relevant words in a sentence (e.g., in "The cat that the dog chased ran," it highlights "cat" and "ran" to understand the action).

• College note: Transformers are now used for everything—protein folding (AlphaFold), image generation (DALL·E), even music. The original 2017 paper ("Attention Is All You Need") is one of the most cited in AI history.

• Hallucination

• Definition: When an LLM generates false or nonsensical information with high confidence, often because it’s extrapolating from patterns in its training data without actual knowledge.
• Example: Ask an LLM "What was the first book written on Mars?" and it might invent a title like "Red Dust Chronicles" with a fake author and plot summary—because it’s seen enough sci-fi books to mimic the structure, even though no such book exists.

• College note: Hallucinations are a major unsolved problem in AI. Some researchers think they’re inherent to how LLMs work (statistical mimicry, not truth-seeking), while others are exploring "retrieval-augmented" models that pull from databases to fact-check.

• Bias (in AI)

• Definition: Systematic errors in an LLM’s outputs that reflect prejudices or imbalances in its training data, often perpetuating stereotypes or excluding marginalized groups.
• Example: If an LLM is trained mostly on news articles from 2010–2020, it might associate "nurse" with "she" and "engineer" with "he" because of historical gender imbalances in those professions. It might also describe a "CEO" as "aggressive" more often than a "teacher," reflecting cultural biases.
• College note: Bias in AI is a hotly debated topic in ethics and policy. Some argue it’s impossible to eliminate bias entirely (since all data is biased), while others focus on "fairness metrics" to audit models. The field of AI alignment studies how to make models reflect human values.

3. Assessment Translation

How this appears on assessments (Grade 10):
- Classroom formative assessments: Short-answer questions, debates, or "explain like I’m 5" prompts. Teachers look for: - Proficient: Clear explanation of next-token prediction with a concrete example (e.g., "The AI doesn’t ‘know’ grammar—it just predicts ‘the’ after ‘sat on’ because that’s the most common pattern in its training data.").
- Developing: Vague or anthropomorphic language ("The AI thinks about the words" or "It’s smart like a human").
- Common distractor: Confusing LLMs with "true" understanding ("The AI understands the meaning of the sentence").

• State standardized tests (e.g., digital literacy or CS assessments):
• Multiple choice: Questions like "Which of the following best describes how a large language model generates text?" with distractors like:
  • ❌ "It searches the internet in real time for answers." (LLMs don’t browse the web.)
  • ❌ "It uses logic and reasoning like a human." (They don’t "reason"—they predict.)
  • ✅ "It predicts the next word based on patterns in its training data."

• Short answer: "Explain why an LLM might generate a false fact about a historical event. Use the term ‘hallucination’ in your answer." (Proficient response ties it to next-token prediction and lack of real-world knowledge.)

• SAT/ACT (if relevant to digital literacy sections):

• Reading comprehension: Passages about AI ethics or technology, with questions testing ability to distinguish between how LLMs work and what they can/can’t do.
• Example prompt: "The author claims that LLMs ‘hallucinate’ facts. Which detail from the passage best supports this claim?" (Correct answer would cite an example of the AI inventing information.)

Model Proficient Response (Short Answer):
Prompt: "Why might an LLM describe a toaster as having ‘feelings’ if asked to write a story about one? Use the terms ‘training data’ and ‘next-token prediction’ in your answer."

Response: An LLM doesn’t actually know what a toaster is—it’s just predicting the next word based on patterns in its training data. If the AI has seen lots of stories where objects are personified (like talking cars or emotional robots), it might predict words like "felt" or "sad" after "toaster" because those are statistically likely in that context. It’s not that the AI thinks toasters are alive; it’s just mimicking the structure of stories it’s read. This is why LLMs can seem creative but also make weird mistakes—they’re good at patterns, not facts.

4. Mistake Taxonomy

Mistake 1: Anthropomorphizing the AI
- Prompt: "Does an LLM like ChatGPT ‘understand’ language? Explain your answer." - Common wrong response: "Yes, because it can write essays and answer questions like a human." - Why it loses credit: The question asks for an explanation of how the AI works, not just its capabilities. This response treats the AI like a person, ignoring the core idea of next-token prediction.
- Correct approach: - Define "understand" (e.g., "Does it grasp meaning, or just mimic patterns?").
- Explain next-token prediction with an example (e.g., "If you type ‘The sky is ___,’ it predicts ‘blue’ not because it knows the sky is blue, but because that’s the most common word in its training data.").
- Conclude: "It simulates understanding but doesn’t truly comprehend language."

Mistake 2: Ignoring Bias in Training Data
- Prompt: "Give one example of how an LLM might reflect bias in its outputs. Explain why this happens." - Common wrong response: "It might say something mean." (Too vague; doesn’t tie to training data.) - Why it loses credit: The question asks for a specific example and an explanation rooted in how the AI works. This response doesn’t connect bias to the model’s training process.
- Correct approach: - Give a concrete example (e.g., "If asked to describe a ‘scientist,’ an LLM might use ‘he’ more often than ‘she’ because older texts used male pronouns for scientists.").
- Explain: "The AI learns from human-written text, which includes historical biases. It doesn’t ‘know’ these are problematic—it just repeats patterns." - Bonus: Mention that fixing this requires curating training data, not just more data.

Mistake 3: Confusing LLMs with Search Engines
- Prompt: "How does an LLM like ChatGPT differ from a search engine like Google?" - Common wrong response: "ChatGPT gives better answers because it’s smarter." (Misses the key difference: prediction vs. retrieval.) - Why it loses credit: The question is about how they work, not which is "better." This response ignores the core mechanism of each tool.
- Correct approach: - Search engine: "Google indexes web pages and retrieves the most relevant ones based on keywords and links." - LLM: "ChatGPT generates new text by predicting the next word in a sequence, without accessing the internet or ‘knowing’ facts." - Key difference: "One finds existing information; the other creates new text based on patterns."

5. Connection Layer

1. Within AI: [Next-token prediction] → [Generative AI for images (DALL·E, Stable Diffusion)]

2. Why it matters: Both LLMs and image generators use neural networks to predict what comes next—but for images, it’s predicting the next pixel (or "patch" of pixels) instead of the next word. Understanding one makes the other clearer: they’re both statistical mimicry, not true creativity.

3. Across subjects: [LLMs and probability] → [Statistics (Bayes’ Theorem, conditional probability)]

4. Why it matters: LLMs are essentially massive probability engines. When an LLM calculates that "mat" is 30% likely after "The cat sat on the ___," it’s using conditional probability—the same math behind spam filters or medical diagnoses. The difference? LLMs do it at a scale no human could.

5. Outside school: [Hallucinations in LLMs] → [Deepfake videos and misinformation]

6. Why it matters: Both LLMs and deepfakes exploit the gap between what seems real and what is real. A deepfake video might show a politician saying something they never said, just like an LLM might invent a fake historical event. In both cases, the technology is convincing but not truthful—and that’s what makes them dangerous. Next time you see a viral "leak" or "quote," you’ll ask: Is this a hallucination?

6. The Stretch Question

"If an LLM is just predicting the next word, why does it sometimes write things that feel deeply original—like a poem that makes you cry or a joke that’s actually funny? Isn’t that more than just statistics?"

Pointer toward the answer:
Originality in humans comes from combining ideas in new ways—so does an LLM’s "creativity." When it writes a sad poem about a robot, it’s not feeling sadness; it’s remixing patterns from thousands of sad poems, robot stories, and emotional metaphors in its training data. The "originality" is an emergent property of scale: with enough data, the combinations become so complex that they seem new. But here’s the twist: humans do this too! Our "original" ideas are often remixes of things we’ve read, seen, or experienced. The difference? We experience the world; LLMs only simulate it. So is the LLM’s creativity "real"? That depends on how you define "real"—and that’s a question philosophers and AI researchers are still arguing about.