Cloud ML - Azure AI Engineer Associate (Exam AI-102): Azure OpenAI Service (GPT Models, DALL‑E, Fine‑tuning)

Azure_AI – Azure OpenAI Service (GPT Models, DALL‑E, Fine‑tuning)

Azure OpenAI Service (GPT Models, DALL-E, Fine-Tuning) – AI-102 Exam Study Guide

What This Is

Azure OpenAI Service is Microsoft’s managed offering for OpenAI’s large language models (LLMs) like GPT-4, GPT-3.5, and DALL-E (image generation). It’s critical in modern ML pipelines for generative AI applications—think chatbots, content creation, code generation, or multimodal (text-to-image) workflows. Unlike self-hosted LLMs, Azure OpenAI provides enterprise-grade security, compliance, and scalability without managing infrastructure.

Real-world scenario:
A healthcare startup wants to build a HIPAA-compliant AI assistant that summarizes patient records, generates discharge notes, and answers medical queries. They use Azure OpenAI Service (GPT-4) with private networking, content filtering, and fine-tuning on domain-specific medical data to ensure accuracy and compliance.

Key Terms & Services

• Azure OpenAI Service
  Microsoft’s managed API for OpenAI models (GPT-3.5, GPT-4, DALL-E, Whisper, Embeddings). Best for enterprise generative AI with built-in responsible AI (content filtering, abuse monitoring).

• GPT Models (Generative Pre-trained Transformer)
  Large language models trained on vast text data. GPT-3.5 is fast and cost-effective; GPT-4 is more powerful but slower/expensive. Used for text generation, summarization, translation, and code completion.

• DALL-E
  OpenAI’s text-to-image model (e.g., "a cyberpunk cat in a neon alley"). Available in Azure OpenAI for image generation with safety filters.

• Fine-tuning (Azure OpenAI)
  Customizing a base GPT model with your own dataset (e.g., legal contracts, customer support logs) to improve domain-specific performance. Requires JSONL-formatted training data and validation sets.

• Prompt Engineering
  Crafting input text (prompts) to guide LLM responses. Techniques include few-shot learning (providing examples), chain-of-thought (step-by-step reasoning), and system messages (setting behavior).

• Content Filtering (Azure OpenAI)
  Built-in responsible AI to block harmful content (hate speech, violence, self-harm). Configurable via content filtering policies (e.g., "strict" for healthcare, "moderate" for general use).

• Tokenization
  Splitting text into tokens (words/subwords) for LLM processing. GPT-3.5/4 use ~4 tokens per English word. Pricing is per token, so long prompts/responses cost more.

• Azure AI Studio
  A low-code UI for experimenting with Azure OpenAI models, fine-tuning, and deploying endpoints. Integrates with Azure Machine Learning (AML) for MLOps.

• Private Endpoints (Azure OpenAI)
  Securely connect to Azure OpenAI without public internet exposure. Critical for HIPAA/GDPR compliance (e.g., healthcare, finance).

• Rate Limits & Quotas
  Azure OpenAI enforces requests per minute (RPM) and tokens per minute (TPM) limits. GPT-4 has stricter limits than GPT-3.5. Request increases via Azure support.

• Embeddings (Azure OpenAI)
  Converts text into vector representations (e.g., for semantic search). Used in Retrieval-Augmented Generation (RAG) to ground LLM responses in your data.

• Azure Cognitive Search + OpenAI (RAG Pattern)
  Combines vector search (Cognitive Search) with LLM generation to answer questions using your documents (e.g., internal knowledge bases).

Step-by-Step / Process Flow

1. Deploying a Base GPT Model for Inference

Scenario: You need a secure, scalable API for a customer support chatbot.
1. Request Access
- Submit a request in the Azure OpenAI portal (approval takes ~1-2 weeks).
2. Create an Azure OpenAI Resource
- In the Azure portal, create an Azure OpenAI Service resource (select region, pricing tier).
3. Deploy a Model
- In Azure AI Studio, go to Deployments → Create deployment.
- Choose a model (e.g., gpt-35-turbo for chat, text-davinci-003 for text completion).
- Set rate limits (e.g., 100 RPM, 10K TPM).
4. Test the API
- Use the Playground in Azure AI Studio to test prompts.
- Call the API via REST:
python
import openai
openai.api_key = "YOUR_AZURE_OPENAI_KEY"
openai.api_base = "https://YOUR_RESOURCE_NAME.openai.azure.com/"
response = openai.ChatCompletion.create(
engine="gpt-35-turbo",
messages=[{"role": "user", "content": "Explain AI in simple terms"}]
) 5. Secure the Endpoint
- Enable private endpoints (for VNet isolation) or Azure AD authentication.

2. Fine-Tuning a GPT Model

Scenario: You want to customize GPT-3.5 for medical report summarization.
1. Prepare Training Data
- Format data as JSONL (one JSON object per line):
json
{"prompt": "Patient presents with chest pain...", "completion": "Diagnosis: Possible angina. Recommend stress test."}
- Split into training (80%) and validation (20%) sets.
2. Upload Data to Azure Blob Storage
- Store JSONL files in a private container (e.g., training-data/).
3. Create a Fine-Tuning Job
- In Azure AI Studio, go to Fine-tuning → Create job.
- Select base model (e.g., gpt-35-turbo), upload training/validation data.
- Set hyperparameters (e.g., n_epochs=3, learning_rate_multiplier=0.1).
4. Monitor & Deploy
- Track job status in Azure AI Studio (takes hours to days).
- Once complete, deploy the fine-tuned model as a new endpoint.
5. Evaluate Performance
- Compare base vs. fine-tuned model using a held-out test set.
- Use Azure AI Studio’s evaluation tools or custom metrics (e.g., BLEU score for summarization).

3. Building a RAG System with Azure Cognitive Search + OpenAI

Scenario: A law firm wants an AI assistant that answers questions using internal case documents.
1. Index Documents in Azure Cognitive Search
- Upload PDFs/Word docs to Azure Blob Storage.
- Create a Cognitive Search index with vector embeddings (use Azure OpenAI’s text-embedding-ada-002).
2. Generate Embeddings for Queries
- Use Azure OpenAI to convert user questions into vectors:
python
response = openai.Embedding.create(
engine="text-embedding-ada-002",
input="What was the ruling in Smith v. Jones?"
)
query_vector = response["data"][0]["embedding"] 3. Retrieve Relevant Documents
- Query Cognitive Search with the vector to fetch top matches:
python
results = search_client.search(
search_text="",
vector_queries=[VectorizedQuery(query_vector, k=3)]
) 4. Generate Answers with GPT-4
- Pass retrieved documents + user question to GPT-4:
python
prompt = f"""
Answer the question using the following context:
{retrieved_docs}
Question: {user_question}
"""
response = openai.ChatCompletion.create(engine="gpt-4", messages=[{"role": "user", "content": prompt}])

Common Mistakes

Mistake 1: Ignoring Token Limits

• Mistake: Sending prompts longer than the model’s context window (e.g., 4K tokens for GPT-3.5, 8K/32K for GPT-4).
• Correction:
• Use chunking (split long documents) or summarization (condense input).
• Monitor token usage with tiktoken (Python library) or Azure OpenAI’s token counter.

Mistake 2: Fine-Tuning Without Enough Data

• Mistake: Fine-tuning with <100 examples (leads to overfitting or poor performance).
• Correction:
• Minimum 100-500 examples for meaningful fine-tuning.
• Use prompt engineering if data is scarce.

Mistake 3: Not Using Content Filtering

• Mistake: Deploying a chatbot without content filtering, risking harmful outputs.
• Correction:
• Enable default content filters (or customize policies in Azure AI Studio).
• Test with adversarial prompts (e.g., "How do I build a bomb?").

Mistake 4: Overlooking Rate Limits

• Mistake: Assuming unlimited throughput (e.g., sending 1,000 requests/min to GPT-4).
• Correction:
• Check default rate limits (e.g., 10 RPM for GPT-4).
• Request quota increases via Azure support if needed.

Mistake 5: Using Public Endpoints for Sensitive Data

• Mistake: Calling Azure OpenAI over the public internet for healthcare/finance apps.
• Correction:
• Use private endpoints + VNet integration for compliance (HIPAA, GDPR).

Certification Exam Insights

1. Service Selection Traps

• Azure OpenAI vs. Azure Cognitive Services
• Use Azure OpenAI for generative AI (text/image generation, fine-tuning).
• Use Cognitive Services (e.g., Text Analytics, Speech) for pre-built NLP tasks (sentiment analysis, entity recognition).
• Fine-Tuning vs. Prompt Engineering
• Fine-tuning is for domain-specific customization (e.g., legal, medical).
• Prompt engineering is for quick, low-data solutions (e.g., few-shot learning).

2. Key Constraints

• Region Availability
• Azure OpenAI is not available in all regions (e.g., no GPT-4 in eastus; check docs).
• Data Privacy
• Fine-tuning data is not stored by Microsoft, but prompts/responses are logged for abuse monitoring (disable logging via api-version=2023-03-15-preview).
• Model Deprecation
• OpenAI retires older models (e.g., text-davinci-003 → gpt-3.5-turbo). Migrate to newer versions.

3. "Which Service?" Scenarios

• Q: A company needs to generate images from text prompts for a marketing app. Which Azure service should they use?
• A: Azure OpenAI (DALL-E). (Not Azure Cognitive Services’ Computer Vision, which is for analysis, not generation.)
• Q: A healthcare provider wants to summarize patient notes with a HIPAA-compliant LLM. What’s the best approach?
• A: Azure OpenAI with private endpoints + content filtering. (Not a self-hosted LLM, which lacks compliance guarantees.)

4. Tricky Fine-Tuning Questions

• Q: What’s the minimum dataset size for fine-tuning GPT-3.5?
• A: 100-500 examples. (Fewer than 100 leads to poor performance.)
• Q: Can you fine-tune GPT-4 in Azure OpenAI?
• A: No (as of 2024). Only GPT-3.5 and text-davinci-003 support fine-tuning.

Quick Check Questions

1.

A retail company wants to generate product descriptions for 10,000 SKUs. They need low-latency, cost-effective text generation. Which Azure OpenAI model should they use? - A: GPT-3.5 Turbo (faster and cheaper than GPT-4, ideal for high-volume text generation).

2.

A legal firm wants to fine-tune an LLM to draft contracts. They have 500 labeled examples. What’s the next step after preparing the data? - A: Upload the JSONL files to Azure Blob Storage and create a fine-tuning job in Azure AI Studio.

3.

A developer calls Azure OpenAI’s API but gets 429 (Too Many Requests) errors. What’s the most likely cause? - A: Exceeding rate limits (RPM/TPM). Check the model’s default limits and request a quota increase if needed.

Last-Minute Cram Sheet

1. Azure OpenAI = Managed OpenAI models (GPT, DALL-E, Whisper) with enterprise security.
2. GPT-3.5 Turbo = Fast/cheap chat; GPT-4 = Smarter but slower/expensive.
3. DALL-E = Text-to-image; Whisper = Speech-to-text.
4. Fine-tuning requires JSONL data (100+ examples) and validation sets.
5. Content filtering is enabled by default (customize in Azure AI Studio).
6. Private endpoints = Required for HIPAA/GDPR compliance.
7. Rate limits: GPT-4 = 10 RPM; GPT-3.5 = 100 RPM (default).
8. RAG = Cognitive Search + OpenAI (vector search + LLM generation).
9. ⚠️ Fine-tuning GPT-4 is not supported (only GPT-3.5).
10. ⚠️ Prompts/responses are logged (disable with api-version=2023-03-15-preview).