Cloud ML - Azure AI Engineer Associate (Exam AI-102): Select the Appropriate Azure AI Service (Cognitive Services vs Applied AI vs Custom Models)

Azure_AI – Select the Appropriate Azure AI Service (Cognitive Services vs Applied AI vs Custom Models)

Azure AI-102 Study Guide: Select the Appropriate Azure AI Service (Cognitive Services vs. Applied AI vs. Custom Models)

What This Is

Choosing the right Azure AI service is critical for balancing cost, latency, accuracy, and maintenance in an ML pipeline. For example: - A retail company needs real-time product recommendations—should they use pre-built Cognitive Services (fast, low-code) or train a custom model (higher accuracy but more effort)? - A healthcare provider must extract text from handwritten prescriptions—Azure AI Document Intelligence (pre-trained) vs. custom OCR (fine-tuned for medical terms)? - A bank wants fraud detection—Anomaly Detector (Cognitive Service) for quick deployment vs. Azure Machine Learning (custom model) for domain-specific rules?

This guide helps you navigate Azure’s AI offerings and pick the right tool for the job.

Key Terms & Services

Azure Cognitive Services

• Azure Cognitive Services (General): Pre-built, no-code/low-code AI APIs for vision, speech, language, and decision-making. Best for quick deployment, minimal ML expertise, and standard use cases (e.g., sentiment analysis, OCR, face detection).
• Example: Text Analytics for sentiment analysis, Computer Vision for image tagging.

• Best for: Fast time-to-market, no training data required, pay-per-use pricing.

• Azure AI Document Intelligence (formerly Form Recognizer): Pre-trained OCR + document understanding (invoices, receipts, IDs). Supports custom models for domain-specific layouts.

• Best for: Extracting structured data from PDFs, forms, and scanned documents without manual labeling.

• Azure Speech Services: Pre-built speech-to-text (STT), text-to-speech (TTS), and speaker recognition.

• Best for: Voice assistants, call center transcription, real-time or batch processing.

• Azure Language Services: Pre-trained NLP models (entity recognition, PII detection, summarization, translation).

• Best for: Chatbots, multilingual support, and text classification without training data.

• Azure Vision Services: Pre-built image/video analysis (object detection, facial recognition, optical character recognition).

• Best for: Moderation, content tagging, and accessibility (e.g., alt-text generation).

• Azure Decision Services (Anomaly Detector, Personalizer, Content Moderator):

• Anomaly Detector: Detects unusual patterns in time-series data (fraud, IoT sensor failures).
• Personalizer: Reinforcement learning for real-time recommendations (e.g., Netflix-style content ranking).
• Content Moderator: Automated filtering of inappropriate text/images.
• Best for: Low-latency decision-making without custom model training.

Azure Applied AI Services (Higher-Level Abstractions)

• Azure Bot Service: Managed bot framework (integrates with Cognitive Services for NLP, QnA Maker for FAQs).

• Best for: Enterprise chatbots with multi-channel support (Teams, Slack, web).

• Azure Cognitive Search (formerly Azure Search): AI-powered search with semantic ranking, OCR, and vector search (RAG-friendly).

• Best for: Enterprise search, document retrieval, and hybrid (keyword + vector) search.

• Azure Metrics Advisor: Time-series anomaly detection with root-cause analysis (e.g., server monitoring, sales trends).

• Best for: IT operations, DevOps, and business metrics (no ML expertise needed).

Azure Custom Models (Azure Machine Learning)

• Azure Machine Learning (Azure ML): End-to-end ML platform for training, deploying, and managing custom models (classic ML, deep learning, LLMs).
• Best for: High-accuracy, domain-specific models (e.g., medical imaging, fraud detection with proprietary data).

• Key features:

  
  
  • AutoML (no-code model training for tabular, text, vision).
  • Designer (drag-and-drop ML pipelines).
  • Compute instances/clusters (GPU/CPU for training).
  • MLOps (CI/CD, model monitoring, A/B testing).

• Azure OpenAI Service: Managed API for GPT-4, DALL·E, and fine-tuning (LLMs).

• Best for: Generative AI, chatbots, summarization, and code generation (requires Microsoft approval for access).

• Azure Custom Vision: No-code/low-code image classification & object detection (fine-tune with your own labeled data).

• Best for: Domain-specific image recognition (e.g., defect detection in manufacturing).

Step-by-Step: How to Choose the Right Azure AI Service

1. Define the Use Case & Requirements

• Ask:
• Is this a standard problem (e.g., sentiment analysis, OCR) or custom (e.g., medical image classification)?
• Do you need real-time (low latency) or batch processing?
• Do you have labeled training data? If not, can you use a pre-trained model?
• What’s the budget? (Cognitive Services = pay-per-use; custom models = training costs + compute).

2. Check if a Pre-Built Cognitive Service Fits

• Use Cognitive Services if:
• The problem is common (e.g., text translation, face detection).
• You don’t have training data (or can’t label it).
• You need fast deployment (hours/days, not weeks).
• You want pay-as-you-go pricing (no upfront costs).
• Example:
• Need to extract text from receipts? → Azure AI Document Intelligence (pre-trained).
• Need sentiment analysis for customer reviews? → Text Analytics (Cognitive Service).

3. Consider Applied AI Services for Higher-Level Workflows

• Use Applied AI if:
• You need pre-integrated solutions (e.g., chatbots, search, anomaly detection).
• You want AI + business logic (e.g., a bot that also checks inventory).
• Example:
• Need a chatbot for customer support? → Azure Bot Service + QnA Maker.
• Need semantic search for internal documents? → Azure Cognitive Search.

4. Train a Custom Model if Pre-Built Options Don’t Fit

• Use Azure ML or Custom Vision if:
• You have domain-specific data (e.g., medical images, legal documents).
• You need higher accuracy than pre-built models.
• You want full control over model architecture (e.g., fine-tuning LLMs).
• Example:
• Need to classify X-rays for diseases? → Azure ML (custom CNN model).
• Need to detect defects in manufacturing? → Custom Vision (fine-tuned model).

5. Evaluate Cost & Scalability

6. Deploy & Monitor

• Cognitive Services: Deploy via REST API (no infrastructure management).
• Applied AI: Deploy via Azure Portal (e.g., Bot Service, Cognitive Search).
• Custom Models: Deploy via Azure ML endpoints (real-time) or Azure Functions (batch).
• Monitoring:
• Cognitive Services: Built-in metrics in Azure Monitor.
• Custom Models: Azure ML model monitoring (data drift, performance decay).

Common Mistakes

Mistake 1: Using Cognitive Services for Custom Problems

• Example: Trying to use Computer Vision’s pre-trained OCR for handwritten medical prescriptions (fails due to domain-specific terms).
• Correction: Use Azure ML + Custom Vision (fine-tune on medical data) or Azure AI Document Intelligence (custom model).

Mistake 2: Over-Engineering with Custom Models

• Example: Building a custom sentiment analysis model when Text Analytics (Cognitive Service) already works.
• Correction: Start with Cognitive Services → only switch to custom if accuracy is insufficient.

Mistake 3: Ignoring Cost for High-Volume Use Cases

• Example: Using Azure OpenAI (GPT-4) for high-volume chatbot responses (expensive per token).
• Correction: Use Cognitive Services (Language Understanding) for simple queries, fall back to OpenAI for complex ones.

Mistake 4: Not Considering Latency Requirements

• Example: Using Azure ML (custom model) for real-time fraud detection (high latency due to model size).
• Correction: Use Anomaly Detector (Cognitive Service) for low-latency or optimize the custom model (quantization, ONNX runtime).

Mistake 5: Forgetting Compliance & Responsible AI

• Example: Using Face API (Cognitive Service) for employee monitoring without bias testing.
• Correction: Use Azure Responsible AI Dashboard to check for fairness, explainability, and privacy risks.

Certification Exam Insights

1. Service Selection Traps

• Trap: "Which service for real-time handwriting recognition?"
• Wrong answer: Azure ML (custom model).
• Right answer: Azure AI Document Intelligence (pre-trained OCR) or Custom Vision (if fine-tuning is needed).
• Trap: "Which service for semantic search with vector embeddings?"
• Wrong answer: Azure Search (basic keyword search).
• Right answer: Azure Cognitive Search (vector search + semantic ranking).

2. Key Constraints to Remember

• Cognitive Services:
• No training data required (but limited customization).
• Rate limits (e.g., 100 calls/min for some APIs).
• Data residency requirements (some services store data temporarily).
• Azure OpenAI:
• Requires Microsoft approval (not available by default).
• Token limits (e.g., 4K tokens for GPT-3.5, 32K for GPT-4).
• Azure ML:
• Compute costs (GPU clusters are expensive).
• Model size limits (e.g., 2GB for real-time endpoints).

3. "Which Service?" Scenarios

Quick Check Questions

Question 1

A retail company wants to automatically tag product images with categories (e.g., "shoes," "electronics"). They have no labeled data but need fast deployment. Which Azure service should they use? ✅ Answer: Azure Computer Vision (Cognitive Service)
✔ Explanation: Pre-trained image tagging requires no training data and deploys in minutes.

Question 2

A healthcare startup needs to extract patient data from handwritten doctor notes. The text includes medical abbreviations not recognized by standard OCR. Which service should they use? ✅ Answer: Azure AI Document Intelligence (custom model) or Azure ML (custom OCR model)
✔ Explanation: Pre-trained OCR (Cognitive Services) won’t handle domain-specific terms; fine-tuning is needed.

Question 3

A bank wants to detect fraudulent transactions in real-time with minimal false positives. They have historical transaction data but no ML team. Which service should they use? ✅ Answer: Azure Anomaly Detector (Cognitive Service) or Azure ML (AutoML for fraud detection)
✔ Explanation: Anomaly Detector is low-code and fast; AutoML is better if they need higher accuracy with labeled data.

Last-Minute Cram Sheet

1. Cognitive Services = pre-built, no training data, pay-per-use (e.g., Text Analytics, Computer Vision).
2. Applied AI = higher-level workflows (e.g., Bot Service, Cognitive Search).
3. Custom Models = Azure ML or Custom Vision (when pre-built isn’t accurate enough).
4. Azure OpenAI = generative AI (GPT-4, DALL·E) but requires Microsoft approval.
5. ⚠️ Cognitive Services have rate limits (e.g., 100 calls/min for some APIs).
6. ⚠️ Azure ML compute costs add up (GPU clusters are expensive; use spot instances for training).
7. ⚠️ Document Intelligence ≠ Computer Vision (Document Intelligence is for forms/receipts, Computer Vision is for general images).
8. ⚠️ Anomaly Detector is for time-series data (not tabular data like fraud transactions—use Azure ML instead).
9. ⚠️ Azure Cognitive Search supports vector search (for RAG and semantic search).
10. ⚠️ Always check data residency requirements (some Cognitive Services store data temporarily).