Cloud ML - Azure AI Engineer Associate (Exam AI-102): Text Analytics (Sentiment, Key Phrases, Entities, PII)

Azure_AI – Text Analytics (Sentiment, Key Phrases, Entities, PII)

Azure AI-102 Study Guide: Text Analytics (Sentiment, Key Phrases, Entities, PII)

What This Is

Text Analytics in Azure refers to pre-built AI services that extract insights from unstructured text—such as sentiment (positive/negative/neutral), key phrases, named entities (people, places, organizations), and personally identifiable information (PII). These services are critical in ML pipelines where raw text (e.g., customer reviews, support tickets, social media) must be structured for downstream tasks like churn prediction, compliance monitoring, or recommendation systems.

Real-world scenario:
A retail company wants to monitor customer sentiment in real-time from support chats and social media. They use Azure AI Language to detect sentiment, extract key phrases (e.g., "slow delivery"), and redact PII (e.g., credit card numbers) before storing logs for compliance. The structured output feeds into a Power BI dashboard and triggers automated workflows (e.g., escalating negative sentiment to a manager).

Key Terms & Services

• Azure AI Language (formerly Text Analytics):
  Azure’s managed NLP service for sentiment analysis, key phrase extraction, named entity recognition (NER), and PII detection. Best for low-code, scalable text processing without training custom models.

• Sentiment Analysis:
  Classifies text as positive, negative, neutral, or mixed (with confidence scores). Used in customer feedback analysis, brand monitoring, and chatbot responses.

• Key Phrase Extraction:
  Identifies important topics in text (e.g., "fast shipping," "poor packaging"). Useful for topic modeling, document summarization, and trend analysis.

• Named Entity Recognition (NER):
  Detects predefined entities (people, locations, organizations, dates, etc.). Critical for knowledge graphs, search enhancement, and compliance (e.g., identifying "Microsoft" as an organization).

• PII (Personally Identifiable Information) Detection:
  Flags sensitive data (SSNs, emails, phone numbers, credit cards) for GDPR/CCPA compliance, data redaction, and secure storage.

• Language Detection:
  Identifies the language of input text (supports 100+ languages). Required before running other text analytics tasks.

• Azure AI Language Custom Features:

• Custom Named Entity Recognition (CNER): Train a model to detect domain-specific entities (e.g., product codes, medical terms).

• Custom Text Classification: Build a custom sentiment or topic classifier for niche use cases (e.g., legal documents).

• Azure Cognitive Search:
  Search-as-a-service that integrates with Text Analytics to enrich search results with sentiment, entities, and key phrases. Used in enterprise search, e-commerce, and knowledge bases.

• Azure Event Hubs + Stream Analytics:
  Real-time pipeline for processing streaming text (e.g., tweets, chat logs) with Text Analytics. Enables live sentiment dashboards and alerting.

• Azure Synapse Analytics:
  Big data analytics service that can batch-process text (e.g., millions of reviews) using Text Analytics via Synapse Pipelines.

• Responsible AI Dashboard (Azure ML):
  Visualizes bias, fairness, and explainability in Text Analytics outputs (e.g., "Does sentiment analysis favor certain demographics?").

Step-by-Step / Process Flow

1. Set Up Azure AI Language

• Create a Language resource in the Azure Portal:
• Choose "Language" under AI + Machine Learning.
• Select pricing tier (Free for testing, Standard for production).
• Note the endpoint URL and API key (needed for authentication).
• Enable features (sentiment, key phrases, entities, PII) in the Language Studio (web UI) or via API.

2. Preprocess Text (Optional but Recommended)

• Clean text (remove HTML tags, special characters, excessive whitespace).
• Detect language (if multilingual) using the Language Detection API.
• Chunk long documents (Text Analytics has a 5,120-character limit per request).

3. Call the Text Analytics API

• Option 1: REST API (Programmatic)
• Use HTTP POST to the endpoint (e.g., https://<your-resource>.cognitiveservices.azure.com/text/analytics/v3.1/sentiment).
• Include the API key in headers (Ocp-Apim-Subscription-Key).
• Example request body (JSON):
  json
{
"documents": [
{ "id": "1", "text": "I love Azure AI! The sentiment analysis is amazing." }
]
}
• Option 2: Azure SDKs (Python, .NET, etc.)
• Install the azure-ai-textanalytics Python package:
  bash
pip install azure-ai-textanalytics

• Authenticate and call the service:
  ```python
  from azure.ai.textanalytics import TextAnalyticsClient
  from azure.core.credentials import AzureKeyCredential

  endpoint = "https://.cognitiveservices.azure.com/" key = "" client = TextAnalyticsClient(endpoint, AzureKeyCredential(key))

  documents = ["I love Azure AI!", "The service is too slow."] response = client.analyze_sentiment(documents) for doc in response:
  print(f"Sentiment: {doc.sentiment} (Confidence: {doc.confidence_scores})") ``` - Option 3: Low-Code (Logic Apps, Power Automate)
  - Use the Text Analytics connector in Power Automate to process emails, forms, or SharePoint data.

4. Process Outputs

• Sentiment: Extract sentiment (positive/negative/neutral) and confidence_scores (0–1).
• Key Phrases: Loop through key_phrases (e.g., ["Azure AI", "sentiment analysis"]).
• Entities: Access entities with category (e.g., "Person"), text (e.g., "Satya Nadella"), and confidence_score.
• PII: Redact or log redacted_text (e.g., "My credit card is [REDACTED]").

5. Store & Use Results

• Batch processing: Store outputs in Azure Blob Storage, Cosmos DB, or SQL Database for analytics.
• Real-time processing: Stream results to Power BI, Event Hubs, or Azure Functions for alerts.
• Custom models: Use Azure ML to fine-tune outputs (e.g., adjust sentiment thresholds for your domain).

6. Monitor & Optimize

• Track API usage in the Azure Portal (costs scale with request volume).
• Enable logging (via Azure Monitor) to debug errors (e.g., rate limits, invalid inputs).
• Optimize costs by:
• Batching requests (up to 1,000 documents per call).
• Using Free tier for testing (5,000 transactions/month).

Common Mistakes

Certification Exam Insights

What the AI-102 Exam Tests

1. Service Selection Traps:
2. "When to use Azure AI Language vs. Azure Cognitive Search?"
   • Use AI Language for text analytics (sentiment, entities, PII).
   • Use Cognitive Search for search enrichment (e.g., adding sentiment to search results).

3. "Azure AI Language vs. Azure Form Recognizer?"

   
   
   • Form Recognizer extracts structured data from documents (invoices, receipts).
   • AI Language processes unstructured text (reviews, emails).

4. Key Constraints:

5. Free tier limits: 5,000 transactions/month (exam may ask about cost optimization).
6. Document size limit: 5,120 characters (know how to handle longer texts).

7. PII detection: Only supports English by default (other languages require custom models).

8. Tricky Scenarios:

9. "A company needs to analyze sentiment in 10,000 customer reviews daily. Which service is most cost-effective?"
   • Answer: Azure AI Language (batch processing) + Azure Synapse Analytics (for storage/analytics).
   • Trap: Suggesting Azure Databricks (overkill for this use case).

10. "How do you redact PII from chat logs before storing them in a database?"

    
    
    • Answer: Use PII detection in AI Language, then Azure Functions to redact and store the cleaned text.
    • Trap: Suggesting Azure Information Protection (this encrypts data, not redacts it).

11. Custom vs. Pre-Built Models:

12. Know when to use pre-built Text Analytics (quick, no training) vs. Custom Text Classification/CNER (domain-specific needs).

Quick Check Questions

Question 1

A healthcare startup needs to extract medical terms (e.g., "hypertension," "ibuprofen") from patient notes. The terms are not recognized by the default Named Entity Recognition (NER) model. Which Azure service should they use?

Answer: Custom Named Entity Recognition (CNER) in Azure AI Language.
Explanation: CNER allows training a model on domain-specific entities (e.g., medical terms), while the pre-built NER only detects generic entities (people, places).

Question 2

A retail company wants to analyze customer sentiment from Twitter in real-time and trigger alerts for negative reviews. Which Azure services should they combine?

Answer: Azure Event Hubs (ingest tweets) + Azure Stream Analytics (process) + Azure AI Language (sentiment) + Azure Logic Apps (alerts).
Explanation: This pipeline streams tweets, analyzes sentiment, and triggers actions (e.g., emails) for negative reviews.

Question 3

A financial services firm must redact PII (e.g., SSNs, credit card numbers) from call center transcripts before storing them. The transcripts are in English and Spanish. What’s the most efficient approach?

Answer: Use Azure AI Language’s PII detection (English) + a custom model for Spanish (if needed) + Azure Functions to redact and store the cleaned text.
Explanation: PII detection supports English by default, but Spanish requires a custom model or translation. Azure Functions automate the redaction process.

Last-Minute Cram Sheet

1. Azure AI Language = Managed NLP for sentiment, key phrases, entities, PII.
2. 5,120-character limit per document ⚠️ – Split long texts before sending.
3. PII detection only supports English by default – Use custom models for other languages.
4. Free tier: 5,000 transactions/month (Standard tier for production).
5. Batch processing: Up to 1,000 documents per API call (cost-efficient).
6. Custom NER (CNER) = Train for domain-specific entities (e.g., medical, legal).
7. Azure Cognitive Search = Enrich search with sentiment/entities (not for raw text analytics).
8. Rate limits: 100 calls/second (Standard tier) ⚠️ – Use retry logic.
9. Language detection first if input is multilingual.
10. ⚠️ PII detection ≠ encryption – Use Azure Information Protection for storage security.