Data Science and Machine Learning 101: Deep Learning and NLP Generative AI and Large Language Models LLMs Prompt Engineering Finetuning RAG Embeddings

What This Is

Generative AI refers to models that create new data—text, code, images, or embeddings—rather than just predict a label. Large Language Models (LLMs) such as GPT‑4, LLaMA, or Claude are transformer‑based generators that can understand prompts, produce coherent language, and be adapted (via prompting, fine‑tuning, or Retrieval‑Augmented Generation) to downstream tasks like drafting customer‑support replies, generating product descriptions, or powering a “chat‑with‑your‑data” analytics assistant.

Key Terms & Formulas

• Transformer Encoder‑Decoder – Architecture that uses self‑attention (Attention(Q,K,V) = softmax(QKᵀ/√d_k) V) to model relationships across all tokens.
• Prompt Engineering – Crafting the input string (system + user messages) to steer the LLM’s output; e.g., "You are a data‑science mentor. Explain k‑means in 2 sentences."
• Fine‑tuning – Updating LLM weights on a domain‑specific dataset; loss typically Cross‑Entropy: L = - Σ y_i log(p_i).
• LoRA (Low‑Rank Adaptation) – Adds trainable low‑rank matrices ΔW = A·B to frozen LLM weights, drastically reducing GPU memory.
• Retrieval‑Augmented Generation (RAG) – Pipeline: (1) embed query, (2) retrieve top‑k documents via similarity search, (3) concatenate docs to prompt, (4) generate answer.
• Embedding – Vector representation e = f(text) (often 768‑dim for BERT‑style models) where cosine similarity sim(e₁,e₂) = (e₁·e₂)/(‖e₁‖‖e₂‖) measures semantic closeness.
• Vector Store (FAISS / Milvus) – Approximate nearest‑neighbor index that supports search(e_q, k) in sub‑millisecond time.
• Instruction Tuning – Training on a mixture of prompts + desired responses to make the model follow human instructions better.
• Hallucination – When the LLM generates plausible‑looking but factually incorrect content; mitigated by grounding (RAG) or post‑hoc verification.
• Temperature (τ) – Sampling hyperparameter; higher τ → more random output. Sampling formula: p_i = softmax(logits_i / τ).
• Top‑p (nucleus) Sampling – Keep smallest set of tokens with cumulative probability ≥ p (e.g., 0.9) before sampling.
• Parameter-Efficient Fine‑Tuning (PEFT) – Techniques like Adapter, Prefix Tuning, or LoRA that add only a few thousand trainable parameters instead of updating the whole model.

Step‑by‑Step / Process Flow

1. Collect & Index Knowledge Base
   python
docs = load_documents('support_kb/') # raw text files
embeddings = model.encode(docs) # sentence‑transformers
index = faiss.IndexFlatIP(embeddings.shape[1]) # inner‑product index
index.add(embeddings)
2. Design Prompt Template
   python
TEMPLATE = """You are a helpful support agent.
Context: {retrieved}
Question: {user_query}
Answer:"""
3. Retrieve Relevant Passages (RAG)
   python
q_emb = model.encode(user_query)
_, I = index.search(q_emb, k=5) # top‑5 docs
retrieved = "\n".join([docs[i] for i in I[0]])
4. Generate Answer with LLM
   python
prompt = TEMPLATE.format(retrieved=retrieved,
user_query=user_query)
response = openai.ChatCompletion.create(
model="gpt-4", messages=[{"role":"user","content":prompt}],
temperature=0.2, top_p=0.95)
5. Post‑process & Validate
6. Run a factuality checker (e.g., a smaller BERT classifier) on the response.
7. If confidence < 0.8 → fallback to “I’m not sure, let me forward to a human.”
8. Iterate & Fine‑tune (optional)
9. Gather user‑rated Q&A pairs.
10. Fine‑tune with LoRA on this dataset (trainer = Trainer(..., peft_config=LoRAConfig(...))).

Common Mistakes

Data Science Interview / Practical Insights

1. Prompt vs. Fine‑tune – Explain when you’d prefer prompt engineering (quick, no GPU) versus fine‑tuning (domain‑specific accuracy, higher compute).
2. RAG vs. End‑to‑End Generation – Discuss trade‑offs: RAG adds latency but reduces hallucination; end‑to‑end is simpler but riskier for factual tasks.
3. Embedding Similarity Metrics – Be ready to compare cosine similarity, Euclidean distance, and inner product; know why cosine is common for normalized vectors.
4. Parameter‑Efficient Fine‑Tuning – Interviewers may ask you to outline LoRA’s update rule (ΔW = A·B) and why it scales to billions of parameters.

Quick Check Questions

1. Scenario: Your chatbot frequently fabricates product specs that aren’t in the knowledge base.
   Answer: Enable Retrieval‑Augmented Generation (RAG) and lower the temperature; grounding forces the model to use real documents.

2. Scenario: You have 200 labeled Q&A pairs and a 7‑B LLM. Training the full model would exceed GPU memory.
   Answer: Use LoRA (or another PEFT method) to fine‑tune only a few thousand parameters, keeping the base frozen.

3. Scenario: After fine‑tuning, validation loss drops but test accuracy stays flat.
   Answer: You’re overfitting; add regularization (e.g., weight decay), increase dropout, or augment the training data.

Last‑Minute Cram Sheet (10 one‑liners)

1. Transformer attention: Attention(Q,K,V)=softmax(QKᵀ/√d_k)·V.
2. Cross‑entropy loss: L = -∑ y_i log(p_i).
3. LoRA update: ΔW = A·B (A∈ℝ^{d×r}, B∈ℝ^{r×d}); r ≪ d.
4. Cosine similarity: sim = (e₁·e₂)/(‖e₁‖‖e₂‖).
5. Temperature scaling: p_i = softmax(logits_i / τ).
6. Top‑p sampling: keep smallest token set with cumulative prob ≥ p (e.g., 0.9).
7. RAG pipeline: Embed → Retrieve → Prompt → Generate.
8. FAISS index type: IndexFlatIP for inner‑product (cosine) search on normalized vectors.
9. ⚠️ Hallucination trap: LLMs are stochastic; never trust a fact without external verification.
10. ⚠️ Token limit trap: Exceeding the model’s max tokens silently truncates the prompt → loss of critical context.