Cloud ML - AWS Certified Machine Learning Engineer – Associate (MLA-C01): Prompt Engineering and RAG Patterns on AWS

AWS_ML – Prompt Engineering and RAG Patterns on AWS

AWS Certified Machine Learning – Specialty: Prompt Engineering & RAG Patterns on AWS

Study Guide for Data Engineers & ML Practitioners

What This Is

Prompt engineering and Retrieval-Augmented Generation (RAG) are techniques to improve LLM outputs by crafting precise instructions (prompts) and grounding responses in external knowledge (retrieval). In AWS, this is critical for building secure, scalable, and cost-effective generative AI applications—like a customer support chatbot that pulls answers from a company’s knowledge base (e.g., PDFs, FAQs, or databases) instead of hallucinating. RAG reduces hallucinations, improves accuracy, and enables private data integration without fine-tuning.

Real-world scenario:
A healthcare provider wants to deploy an LLM-powered clinical assistant that answers doctor queries using internal medical guidelines (stored in S3 as PDFs) while complying with HIPAA. They use Amazon Bedrock for the LLM, Amazon OpenSearch Serverless for vector search, and AWS Lambda to orchestrate retrieval and prompt construction.

Key Terms & Services

AWS-Specific Services

• Amazon Bedrock
  AWS’s fully managed service for building generative AI apps using foundation models (FMs) from Anthropic, Meta, AI21, and Amazon. Supports RAG, fine-tuning, and inference with built-in security (VPC, IAM, KMS). Best for enterprise-grade LLM deployments without managing infrastructure.

• Amazon OpenSearch Serverless (Vector Engine)
  A serverless vector database for semantic search and RAG. Stores embeddings (from models like Titan Embeddings or Hugging Face) and performs k-NN similarity searches at scale. Ideal for low-latency retrieval in chatbots or recommendation systems.

• Amazon Kendra
  AWS’s enterprise search service with ML-powered ranking and natural language understanding (NLU). Supports structured (databases) and unstructured (PDFs, HTML) data with pre-built connectors (S3, SharePoint, Salesforce). Best for document-heavy RAG where keyword + semantic search is needed.

• AWS Lambda
  Serverless compute for running prompt engineering logic (e.g., formatting retrieved context into a prompt). Scales automatically and integrates with Bedrock, OpenSearch, and API Gateway for real-time RAG pipelines.

• Amazon SageMaker JumpStart
  A model hub with pre-trained models (e.g., Flan-T5, Falcon) and one-click deployments. Useful for fine-tuning embeddings or hosting custom RAG models if Bedrock’s FMs don’t meet requirements.

• Amazon Titan Embeddings
  AWS’s foundation embedding model (via Bedrock) that converts text into vector representations for semantic search. Optimized for low-latency retrieval in RAG pipelines.

• AWS Glue / Amazon Athena
  ETL (Glue) and serverless SQL query (Athena) services for preprocessing documents (e.g., chunking PDFs, cleaning text) before indexing in OpenSearch/Kendra.

General Concepts

• Prompt Engineering
  The art of crafting input instructions to guide LLMs toward desired outputs. Techniques include:
• Zero-shot prompting (no examples, just instructions).
• Few-shot prompting (providing 2–3 examples).
• Chain-of-Thought (CoT) (asking the model to "think step-by-step").

• Role prompting (e.g., "You are a medical expert answering patient questions").

• Retrieval-Augmented Generation (RAG)
  A pattern where an LLM retrieves relevant context (e.g., documents, database records) before generating a response. Steps:

• Index documents as vectors (embeddings).
• Retrieve top-k relevant chunks via similarity search.
• Augment the prompt with retrieved context.

• Generate a response using the LLM.

• Chunking Strategy
  How you split documents into smaller pieces for indexing. Common methods:

• Fixed-size chunks (e.g., 512 tokens).
• Semantic chunking (split at paragraph/sentence boundaries).

• Overlapping chunks (to avoid cutting off context).

• Embedding Model
  A model that converts text into dense vector representations (e.g., 1536-dimensional vectors for Titan Embeddings). The quality of embeddings directly impacts RAG performance.

• Vector Database
  A database optimized for storing and querying embeddings (e.g., OpenSearch, Pinecone, Weaviate). Supports approximate nearest neighbor (ANN) search for fast retrieval.

• Hallucination
  When an LLM generates false or unsupported information. RAG reduces hallucinations by grounding responses in retrieved facts.

Step-by-Step: Building a RAG Pipeline on AWS

Scenario:

Deploy a customer support chatbot that answers questions using a company’s internal knowledge base (PDFs in S3).

Step 1: Preprocess & Index Documents

1. Store documents in S3 (e.g., s3://company-knowledge-base/).
2. Use AWS Glue or Lambda to:
3. Extract text from PDFs (e.g., with Amazon Textract or PyPDF2).
4. Clean text (remove headers, footers, special characters).
5. Chunk documents (e.g., 512-token chunks with 10% overlap).
6. Generate embeddings using Titan Embeddings (via Bedrock) or a SageMaker-hosted model.
7. Index embeddings in OpenSearch Serverless or Amazon Kendra:
8. For OpenSearch: Create a vector index with knn_vector field.
9. For Kendra: Use the S3 connector and enable semantic search.

Step 2: Build the Retrieval Layer

1. Set up a retrieval API (e.g., API Gateway + Lambda):
2. Lambda receives a user query (e.g., "How do I reset my password?").
3. Converts the query into an embedding (using Titan Embeddings).
4. Queries OpenSearch/Kendra for top-k relevant chunks.
5. Format the retrieved context into a prompt (e.g., using few-shot examples).

Step 3: Generate Responses with Bedrock

1. Call Amazon Bedrock with:
2. The retrieved context (e.g., "Relevant documents: [chunk1, chunk2]").
3. A prompt template (e.g., "Answer the question using only the provided context. If unsure, say 'I don’t know.' Question: {query}").
4. Stream the response back to the user via API Gateway.

Step 4: Monitor & Optimize

1. Log prompts/responses in Amazon CloudWatch for auditing.
2. Use SageMaker Model Monitor to detect drift (e.g., if retrieval quality degrades).
3. A/B test prompts using Amazon SageMaker Experiments.

Common Mistakes

Mistake 1: Using the Wrong Retrieval Service

• Mistake: Choosing Amazon Kendra for a high-scale, low-latency RAG pipeline (Kendra is optimized for enterprise search with NLU, not pure vector search).
• Correction: Use OpenSearch Serverless for vector-based RAG (faster, cheaper for large-scale retrieval). Use Kendra if you need hybrid search (keyword + semantic) or pre-built connectors (e.g., SharePoint).

Mistake 2: Poor Chunking Strategy

• Mistake: Using fixed-size chunks without overlap, leading to cut-off context (e.g., a sentence split mid-way).
• Correction: Use semantic chunking (split at paragraph/sentence boundaries) and 10–20% overlap to preserve context.

Mistake 3: Ignoring Embedding Model Quality

• Mistake: Using a cheap but weak embedding model (e.g., a small Hugging Face model) for retrieval, leading to poor search results.
• Correction: Use Titan Embeddings (optimized for AWS) or a fine-tuned model (e.g., via SageMaker) for domain-specific data.

Mistake 4: Not Optimizing Prompts for RAG

• Mistake: Sending raw retrieved chunks to the LLM without formatting, causing noisy or irrelevant responses.
• Correction: Use prompt templates with:
• Clear instructions (e.g., "Answer using only the provided context").
• Few-shot examples (e.g., "Example 1: Question: X, Answer: Y").
• Role prompting (e.g., "You are a customer support agent").

Mistake 5: Overlooking Cost in RAG Pipelines

• Mistake: Using Bedrock’s most expensive model (e.g., Claude 3 Opus) for every query, even simple ones.
• Correction: Use model routing:
• Simple queries → Cheaper model (e.g., Claude Instant).
• Complex queries → Expensive model (e.g., Claude 3 Sonnet).

Certification Exam Insights

1. Service Selection Traps

• OpenSearch vs. Kendra for RAG:
• OpenSearch = Pure vector search, low-latency, scalable.
• Kendra = Hybrid search (keyword + semantic), pre-built connectors, better for enterprise document search.

• Exam trap: If the question mentions PDFs in S3 + semantic search, pick Kendra. If it’s high-scale vector search, pick OpenSearch.

• Bedrock vs. SageMaker for RAG:

• Bedrock = Fully managed, no infrastructure, built-in security.
• SageMaker = Custom models, fine-tuning, more control.
• Exam trap: If the question asks for quick deployment with minimal code, pick Bedrock. If it mentions custom embeddings, pick SageMaker.

2. Key Constraints

• Bedrock model limits:
• Input token limits (e.g., Claude 3 Haiku: 200K tokens, Sonnet: 200K tokens).
• Output token limits (e.g., 4K for some models).

• Exam trap: If a question asks about long documents, check if the model supports the required token length.

• OpenSearch Serverless limits:

• Max vector dimensions: 16,000 (Titan Embeddings uses 1,536).
• Indexing throughput: ~10K vectors/sec (burstable).
• Exam trap: If a question mentions millions of vectors, ensure OpenSearch can scale (or suggest partitioning).

3. "Which Service?" Scenarios

Quick Check Questions

Question 1

A fintech company wants to build a RAG-powered fraud detection assistant that answers analyst queries using internal transaction logs (stored in DynamoDB). The solution must support low-latency retrieval and scale to millions of records. Which AWS service should they use for the retrieval layer? - A) Amazon Kendra - B) Amazon OpenSearch Serverless - C) Amazon Aurora PostgreSQL with pgvector - D) AWS Glue

Answer: B) Amazon OpenSearch Serverless
OpenSearch is optimized for low-latency vector search at scale, while Kendra is better for document search with NLU. Aurora + pgvector is an option but requires more management.

Question 2

A healthcare startup is using Amazon Bedrock for a clinical decision support tool. They want to reduce hallucinations by grounding responses in internal medical guidelines (PDFs in S3). Which two services should they use to implement RAG? (Select TWO.) - A) Amazon Textract - B) Amazon OpenSearch Serverless - C) Amazon SageMaker Ground Truth - D) AWS Lambda - E) Amazon Comprehend

Answer: A) Amazon Textract and B) Amazon OpenSearch Serverless
Textract extracts text from PDFs, and OpenSearch indexes embeddings for retrieval. Lambda (D) is needed for orchestration but isn’t one of the two core services here.

Question 3

A company is fine-tuning a custom embedding model for their domain-specific RAG pipeline. They need full control over the training process and want to deploy the model behind a secure endpoint. Which AWS service should they use? - A) Amazon Bedrock - B) Amazon SageMaker - C) AWS Lambda - D) Amazon OpenSearch

Answer: B) Amazon SageMaker
Bedrock doesn’t support custom embedding models, and SageMaker provides fine-tuning + deployment capabilities. Lambda is for orchestration, not training.

Last-Minute Cram Sheet

1. RAG = Retrieval (OpenSearch/Kendra) + Generation (Bedrock/SageMaker).
2. OpenSearch Serverless = Best for high-scale vector search; Kendra = Best for enterprise document search with NLU.
3. Titan Embeddings (via Bedrock) = AWS’s default embedding model for RAG.
4. Chunking strategy matters: Use semantic chunking + overlap to avoid cut-off context.
5. Prompt templates reduce hallucinations: Always include "Answer using only the provided context."
6. Bedrock model limits: Check input/output token limits (e.g., Claude 3 Haiku: 200K input tokens).
7. OpenSearch Serverless max vector dimensions: 16,000 (Titan uses 1,536).
8. ⚠️ Bedrock doesn’t support custom embeddings—use SageMaker instead.
9. ⚠️ Kendra is not a vector database—it’s a hybrid search engine with NLU.
10. Cost optimization: Use model routing (cheap model for simple queries, expensive for complex).