TECH **Python for Data Science: Data Types, Typecasting, and Variables**

Python for Data Science: Data Types, Typecasting, and Variables

A Hyper-Practical, Zero-Fluff Study Guide

1. What This Is & Why It Matters

You’re building a data pipeline that processes customer transactions. Your script reads a CSV file, cleans the data, and calculates average spending. Suddenly, your code crashes with:

TypeError: unsupported operand type(s) for +: 'str' and 'float'

Why? Because the CSV parser read all values as strings ("12.99"), but your math operations expected floats (12.99). This is a data type mismatch—one of the most common (and avoidable) bugs in data science.

In Python for Data Science, data types, typecasting, and variables are the foundation of everything: - Data types determine what operations you can perform (e.g., you can’t multiply a string by a number).
- Typecasting lets you convert between types (e.g., turning "5" into 5 for calculations).
- Variables are how you store and reference data (e.g., customer_age = 30).

Real-world scenario:
You inherit a legacy script that loads a dataset, but the "price" column is stored as strings. Your boss asks for a 10% price increase, but the script fails because "$19.99" * 1.1 is invalid. Fixing this requires understanding data types and typecasting.

If you ignore this, you’ll waste hours debugging silent failures (e.g., "5" + 3 returns "53", not 8), or worse—your model will train on garbage data (e.g., treating "N/A" as a number).

2. Core Concepts & Components

1. Numeric Types

• int: Whole numbers (e.g., 42, -3).
• Production insight: Use int for counts (e.g., rows in a dataset). Avoid floats for IDs (e.g., 1.0 vs 1 can cause SQL join failures).
• float: Decimal numbers (e.g., 3.14, -0.001).
• Production insight: Floats are imprecise (e.g., 0.1 + 0.2 == 0.3 returns False). Use decimal.Decimal for financial calculations.
• complex: Numbers with imaginary parts (e.g., 3 + 4j).
• Production insight: Rare in data science, but used in signal processing or physics simulations.

2. Boolean (bool)

• True or False.
• Production insight: Used in filtering (e.g., df[df["active"] == True]). Remember: 1 == True and 0 == False (but don’t rely on this—explicit is better).

3. Strings (str)

• Text data (e.g., "hello", '42').
• Production insight: Strings are immutable (can’t be changed in-place). Use .replace() or f-strings for formatting (e.g., f"User {id}").

4. Sequences

• list: Ordered, mutable collections (e.g., [1, 2, 3]).
• Production insight: Lists are flexible but slow for large datasets. Use numpy.array or pandas.Series for numerical data.
• tuple: Ordered, immutable collections (e.g., (1, 2, 3)).
• Production insight: Use tuples for fixed data (e.g., coordinates (lat, lon)). Faster than lists for iteration.
• range: Immutable sequence of numbers (e.g., range(0, 10)).
• Production insight: Memory-efficient for loops (doesn’t store all numbers in memory).

5. Mappings

• dict: Key-value pairs (e.g., {"name": "Alice", "age": 30}).
• Production insight: Dictionaries are O(1) for lookups. Use for JSON-like data or column mappings (e.g., {"price": "cost"}).

6. Sets

• set: Unordered, unique elements (e.g., {1, 2, 3}).
• Production insight: Use for deduplication (e.g., set(df["category"])) or membership tests (e.g., if "fraud" in transaction_types).

7. None (NoneType)

• Represents "no value" (e.g., x = None).
• Production insight: Use is None (not == None) to check for missing data. Common in pandas (df["column"].isna()).

8. Typecasting (Explicit Conversion)

• Converting between types (e.g., int("5"), str(3.14)).
• Production insight: Always validate before casting (e.g., if value.isdigit(): int(value)). Silent failures (e.g., int("5.5")) crash your pipeline.

9. Variables

• Names that reference data (e.g., x = 10).
• Production insight:
  • Use snake_case (e.g., customer_age, not CustomerAge).
  • Avoid single-letter names (e.g., x) except in loops.
  • Variables are references, not copies (e.g., a = b means a and b point to the same object).

3. Step-by-Step Hands-On: Cleaning a Messy Dataset

Prerequisites

• Python 3.8+ installed.
• pandas installed (pip install pandas).
• A CSV file with messy data (e.g., "price" as strings with $ signs).

Task

Clean a dataset where: - The "price" column is stored as strings (e.g., "$19.99").
- The "quantity" column has missing values (NaN).
- The "date" column is a string (e.g., "2023-01-15").

Steps

1. Load the data

import pandas as pd

# Load the CSV (replace with your file path)
df = pd.read_csv("transactions.csv")
print(df.head())

Expected output:

   id   price  quantity        date
0   1  $19.99       2.0  2023-01-15
1   2   $5.00       NaN  2023-01-16
2   3  $12.50       1.0  2023-01-17

2. Clean the "price" column (string → float)

# Remove $ and convert to float
df["price"] = df["price"].str.replace("$", "").astype(float)
print(df["price"].head())

Expected output:

0    19.99
1     5.00
2    12.50
Name: price, dtype: float64

3. Handle missing values in "quantity" (NaN → 0)

# Fill NaN with 0 (or use .fillna(1) for default quantity)
df["quantity"] = df["quantity"].fillna(0).astype(int)
print(df["quantity"].head())

Expected output:

0    2
1    0
2    1
Name: quantity, dtype: int64

4. Convert "date" to datetime

# Convert string to datetime
df["date"] = pd.to_datetime(df["date"])
print(df["date"].head())

Expected output:

0   2023-01-15
1   2023-01-16
2   2023-01-17
Name: date, dtype: datetime64[ns]

5. Calculate total revenue (price × quantity)

# Multiply price by quantity (now both are numeric)
df["revenue"] = df["price"] * df["quantity"]
print(df[["price", "quantity", "revenue"]].head())

Expected output:

   price  quantity  revenue
0  19.99         2    39.98
1   5.00         0     0.00
2  12.50         1    12.50

6. Verify no type errors

# Check dtypes
print(df.dtypes)

Expected output:

id            int64
price       float64
quantity      int64
date    datetime64[ns]
revenue     float64
dtype: object

4. ? Production-Ready Best Practices

Security

• Never hardcode sensitive data (e.g., api_key = "12345"). Use environment variables (os.getenv("API_KEY")) or secrets managers.
• Validate inputs: If a function expects an int, check isinstance(value, int) before processing.

Cost Optimization

• Use efficient data types:
• For large datasets, convert float64 to float32 (halves memory usage).
• Use category dtype for strings with few unique values (e.g., "male", "female").
• Avoid unnecessary copies: Use df.copy() only when needed (e.g., before modifying a subset).

Reliability & Maintainability

• Explicit > implicit: Prefer int("5") over int(5.0) (even if both work).
• Document assumptions: Add comments like # price: float (e.g., 19.99) to column descriptions.
• Use type hints (Python 3.5+): python def calculate_revenue(price: float, quantity: int) -> float:
return price * quantity

Observability

• Log type mismatches: Add warnings for unexpected types: python if not isinstance(price, (int, float)):
logging.warning(f"Invalid price type: {type(price)}")
• Monitor memory usage: Use sys.getsizeof() to debug large objects.

5. ⚠️ Common Mistakes & Traps

6. ? Exam/Certification Focus

Typical Question Patterns

1. Typecasting errors:
2. "What is the output of int("3.14")?" → ValueError (can’t convert float string to int directly).

3. "How do you convert "5" to an integer?" → int("5") (not float("5")).

4. Data type properties:

5. "Which is mutable: list or tuple?" → list.

6. "What is the difference between == and is?" → == checks value, is checks memory address.

7. Pandas-specific:

8. "How do you convert a column to datetime?" → pd.to_datetime(df["column"]).
9. "What dtype should you use for a column with 3 unique string values?" → category.

Key ⚠️ Trap Distinctions

• NaN vs None:
• NaN is a float (np.nan), None is NoneType.
• pd.isna() detects both, but == None only detects None.
• astype() vs to_numeric():
• astype(float) fails on "$19.99".
• pd.to_numeric(df["column"], errors="coerce") converts invalid values to NaN.

Scenario-Based Question

"You have a DataFrame with a "user_id" column stored as strings (e.g., "1001"). You need to merge it with another DataFrame where "user_id" is an integer. What’s the most efficient way to handle this?"

Answer:

df1["user_id"] = df1["user_id"].astype(int)  # or pd.to_numeric(df1["user_id"])
df_merged = pd.merge(df1, df2, on="user_id")

Why? Avoids type mismatches during merge and is faster than converting during the merge.

7. ? Hands-On Challenge

Challenge: Write a function that takes a list of mixed types ([1, "2", 3.0, "4.5", None]) and returns a new list with all values converted to floats. Skip None values.

Solution:

def convert_to_floats(mixed_list):
    return [float(x) for x in mixed_list if x is not None]

# Test
print(convert_to_floats([1, "2", 3.0, "4.5", None]))  # [1.0, 2.0, 3.0, 4.5]

Why it works: - float(x) handles int, str, and float inputs.
- if x is not None skips None values.

8. ? Rapid-Reference Crib Sheet

Data Types

Typecasting

Pandas-Specific

⚠️ Exam Traps

• "5" + 3 → "53" (not 8).
• int("5.5") → ValueError (use float first).
• df["col"].isna() detects NaN and None; == None only detects None.
• a = b = [] creates one list referenced by both a and b.

9. ? Where to Go Next

1. Python Data Types (Official Docs) – Deep dive into built-in types.
2. Pandas User Guide: Working with Text Data – String operations and cleaning.
3. Real Python: Python Data Types – Practical examples and exercises.
4. Python Type Hints (PEP 484) – For production-grade code.