Convert 'Date' to datetime and set as index

Study Guide: Introduction to Data Science – Pandas and Matplotlib (Grade 10, Computer Science/ICT)

1. The Driving Question

"If you had a spreadsheet with 10,000 rows of messy data—like every pizza order from a city in one year—how would you find out which toppings are most popular, or whether people order more pepperoni on weekends? And how could you turn those answers into a graph that actually makes sense to someone who isn’t a data scientist?"

This isn’t just about coding—it’s about asking questions of data, cleaning up the mess, and then telling a story with pictures that anyone can understand.

2. The Core Idea – Built, Not Listed

Imagine you’re running a school bake sale, and you’ve tracked every sale for a month in a giant spreadsheet: the item sold, the price, the time of day, and whether the buyer was a student or a teacher. The spreadsheet is a disaster—some prices are written as "$2.50," others as "2.5," and one row just says "free (mom’s friend)." Pandas is like a super-powered spreadsheet assistant that can: - Clean up the mess (fix "$2.50" vs. "2.5" so you can actually add them up).
- Answer questions fast (e.g., "How many cookies were sold before noon?" or "Did teachers spend more than students?").
- Group and summarize (e.g., "What’s the average price for each type of item?").

Once you’ve got the answers, Matplotlib turns them into graphs—like a bar chart showing which item sold the most, or a line graph tracking sales over time. The goal isn’t just to make a graph; it’s to make one that tells the right story (e.g., a pie chart might be terrible for comparing sales over time, but great for showing what fraction of sales were cookies vs. brownies).

Key Vocabulary:
1. DataFrame (Pandas)
- Definition: A 2D table (like a spreadsheet) where you can store, filter, and analyze data using rows and columns.
- Example: A DataFrame of NBA player stats with columns for "Points per Game," "Team," and "Position"—you could filter it to show only players who average >20 points.
- Grade 10 Note: In college, DataFrames get more powerful (e.g., handling missing data automatically or merging datasets with different structures).

1. Series (Pandas)
2. Definition: A single column of data in a DataFrame (like one list of numbers or text).
3. Example: The "Temperature" column in a weather dataset for a month—just a list of daily highs, but you can find the average or plot it over time.

4. Grade 10 Note: In advanced stats, Series can hold complex data types (e.g., time-series data with timestamps).

5. Plot (Matplotlib)

6. Definition: A visual representation of data (e.g., bar chart, line graph, scatter plot).
7. Example: A scatter plot of video game ratings vs. price to see if more expensive games tend to be rated higher.

8. Grade 10 Note: In data science, plots are often interactive (e.g., zooming in on a graph in a web app).

9. Index (Pandas)

10. Definition: The "label" for each row in a DataFrame (like a row number, but you can customize it).
11. Example: In a library book checkout dataset, the index could be the book’s ISBN number instead of row numbers, so you can look up a book directly.
12. Grade 10 Note: In databases, indexes are used to speed up searches (like how a book’s index helps you find a topic faster).

3. Assessment Translation

How This Appears on Assessments:
- Classroom (Formative): Short coding tasks (e.g., "Load this CSV file and calculate the average of column X") or debugging exercises (e.g., "This code throws an error—fix it").
- State/ICT Standards: Multiple-choice questions on data types (e.g., "Which Pandas function would you use to count missing values?") or graph interpretation (e.g., "What does this line graph suggest about sales trends?").
- Project-Based: A mini-data analysis (e.g., "Analyze this dataset of YouTube video stats and create a graph showing the relationship between views and likes").

Distractor Patterns in Multiple Choice:
- Confusing similar functions: df.mean() vs. df.median() vs. df.describe().
- Misreading graphs: Assuming correlation = causation (e.g., "Ice cream sales and drowning incidents both rise in summer—does ice cream cause drowning?").
- Ignoring data cleaning: Questions where the "correct" answer assumes the data is already clean (e.g., forgetting to handle missing values).

Proficient vs. Developing Responses:
| Proficient | Developing | |----------------|----------------| | Code: Uses df.dropna() to clean missing data before calculating averages. | Code: Ignores missing data, leading to incorrect averages. | | Graph: Chooses a scatter plot to show relationship between two variables (e.g., study hours vs. test scores). | Graph: Uses a pie chart for the same data, which doesn’t show correlation. | | Explanation: Writes, "The line graph shows a trend, but we can’t prove causation without more data." | Explanation: Says, "The graph proves that more study hours cause higher scores." |

Model Proficient Response (Short Answer):
Prompt: "Given a DataFrame df with columns ['Date', 'Temperature', 'Rainfall'], write code to plot a line graph of Temperature over time. Label the axes and add a title."

import matplotlib.pyplot as plt

# Convert 'Date' to datetime and set as index
df['Date'] = pd.to_datetime(df['Date'])
df.set_index('Date', inplace=True)

# Plot Temperature
df['Temperature'].plot(kind='line', color='red')
plt.xlabel('Date')
plt.ylabel('Temperature (°F)')
plt.title('Daily Temperature Over Time')
plt.show()

Why it’s proficient: - Handles date formatting (a common stumbling block).
- Labels axes and adds a title (often forgotten).
- Uses inplace=True to avoid creating a new DataFrame unnecessarily.

4. Mistake Taxonomy

Mistake 1: The "Copy-Paste Error"
- Prompt: "Write code to calculate the average of the 'Price' column in a DataFrame sales_df." - Common Wrong Response: python sales_df['Price'].mean - Why It Loses Credit: - Missing parentheses (mean() is a function, not a property).
- No error handling (e.g., if 'Price' has non-numeric values like "$10").
- Correct Approach: python # Clean data first (remove $ signs, convert to float) sales_df['Price'] = sales_df['Price'].replace('[\$,]', '', regex=True).astype(float) # Calculate mean avg_price = sales_df['Price'].mean() print(f"Average price: ${avg_price:.2f}")

Mistake 2: The "Graph Choice Fail"
- Prompt: "Create a graph to show the relationship between 'Hours Studied' and 'Test Score' in a dataset." - Common Wrong Response: python df.plot(kind='pie') # Pie chart for two continuous variables - Why It Loses Credit: - Pie charts are for parts of a whole, not relationships between variables.
- No labels or title (unreadable graph).
- Correct Approach: python df.plot(kind='scatter', x='Hours Studied', y='Test Score', color='blue') plt.xlabel('Hours Studied') plt.ylabel('Test Score') plt.title('Study Time vs. Test Performance') plt.show()

Mistake 3: The "Data Leak"
- Prompt: "Calculate the average 'Age' of customers who bought a product, but first remove rows where 'Age' is missing." - Common Wrong Response: python avg_age = df['Age'].mean() # Calculates mean before dropping missing values df.dropna(subset=['Age'], inplace=True) - Why It Loses Credit: - The mean is calculated before dropping missing values, so it’s incorrect.
- Order of operations matters! - Correct Approach: python df_clean = df.dropna(subset=['Age']) # Drop missing values first avg_age = df_clean['Age'].mean() print(f"Average age: {avg_age:.1f} years")

5. Connection Layer

1. Within Computer Science:
   [Pandas DataFrames] → [SQL Databases]

2. Why: Both use tables with rows and columns, but SQL is for storing data (like a library’s book catalog), while Pandas is for analyzing it (like figuring out which books are checked out most often). Understanding DataFrames makes SQL queries (e.g., SELECT * FROM books WHERE genre = 'Sci-Fi') feel like a natural extension.

3. Across Subjects:
   [Matplotlib Graphs] → [Statistics (Box Plots, Histograms)]

4. Why: A box plot in Matplotlib isn’t just a pretty picture—it’s a way to visualize median, quartiles, and outliers (key stats concepts). If you’ve made one in Python, you’ll recognize it instantly in a math class when your teacher says, "This box plot shows the spread of test scores."

5. Outside School:
   [Data Cleaning] → [Fantasy Sports (Drafting Players)]

6. Why: Fantasy football apps use data science to predict player performance—but the raw data is messy (e.g., "Injured (knee)" vs. "Out for season"). Cleaning and analyzing data (like Pandas does) is how apps decide whether to draft a player or not. Next time you see a "sleeper pick," remember: someone wrote code like yours to find it.

6. The Stretch Question

"If you had a dataset of every song on Spotify (title, artist, duration, popularity, etc.), how would you design a graph to answer: ‘Do longer songs tend to be more popular?’ What would the graph not tell you—and what other data would you need to prove causation?"

Pointer Toward the Answer:
- A scatter plot of song duration vs. popularity might show a trend (or no trend at all). But even if longer songs are more popular, that doesn’t mean length causes popularity—maybe artists with longer songs also have better marketing, or maybe listeners just prefer certain genres that happen to have longer songs.
- To dig deeper, you’d need data on genre, artist fame, release date, and even lyrics (e.g., do songs with certain words get more streams?). This is how real data scientists avoid "correlation ≠ causation" traps—by asking, "What else could explain this pattern?"

Tone Note: For Grade 10, this guide assumes familiarity with Python basics (loops, lists) but not prior data science experience. The analogies (bake sale, fantasy sports) are chosen to feel relevant to teens while mapping cleanly to the concepts. The stretch question invites debate—exactly what a curious student would enjoy.