Cloud ML - Azure AI Engineer Associate (Exam AI-102): Face API (Detection, Verification, Identification, PII considerations)

Azure_AI – Face API (Detection, Verification, Identification, PII considerations)

Azure AI-102 Study Guide: Face API (Detection, Verification, Identification, PII Considerations)

What This Is

The Azure Face API is a cloud-based service that detects, recognizes, and analyzes human faces in images or video streams. It’s critical in AI pipelines for identity verification, access control, emotion analysis, and privacy-compliant facial recognition. A real-world scenario: A bank uses Face API to verify customer identities during mobile onboarding by comparing a selfie with a government-issued ID, while ensuring compliance with PII (Personally Identifiable Information) regulations like GDPR.

Key Terms & Services

• Azure Face API: Microsoft’s managed service for face detection, verification, identification, and emotion analysis. Best for low-code facial recognition with built-in compliance controls.
• Face Detection: Locates human faces in an image and returns bounding boxes, landmarks (eyes, nose, mouth), and attributes (age, gender, emotion).
• Face Verification: Compares two faces (e.g., selfie vs. ID photo) to determine if they belong to the same person. Returns a confidence score (0–1).
• Face Identification: Matches a detected face against a pre-enrolled database (person group) to find the best match. Used in access control or watchlists.
• Person Group: A database of known faces (e.g., employees, VIP customers) used for identification. Must be trained before use.
• Large Person Group: A scalable version of Person Group (supports up to 1 million faces, vs. 1,000 for standard groups).
• Face List: A static collection of faces (e.g., for similarity search) without identity labels. Useful for duplicate detection.
• PII (Personally Identifiable Information): Data that can identify an individual (e.g., face images, biometric templates). Azure Face API stores face data temporarily (24h by default) unless explicitly saved.
• GDPR / CCPA Compliance: Regulations requiring consent, data minimization, and right to erasure for facial recognition. Azure Face API provides data residency controls (e.g., storing data in EU-only regions).
• Confidence Threshold: A minimum score (0–1) required for a match in verification/identification. Default is 0.5, but adjust based on use case (e.g., 0.7 for high-security access).
• Face Attributes: Optional metadata (age, gender, emotion, facial hair, glasses). Disabled by default for privacy reasons.
• Azure Cognitive Services: A suite of AI services (including Face API) that can be deployed as REST APIs or containers for edge scenarios.

Step-by-Step / Process Flow

1. Set Up Face API & Authenticate

• Create an Azure Cognitive Services resource in the portal (select Face API).
• Copy the endpoint and API key (or use Azure AD authentication for production).
• Install the azure-cognitiveservices-vision-face SDK (Python) or use REST API calls.

2. Detect Faces & Extract Attributes

• Call face.detect_with_stream() (Python) or POST /detect (REST) with an image.
• Specify parameters:
• returnFaceId=True (required for verification/identification).
• returnFaceLandmarks=True (for facial landmarks).
• returnFaceAttributes=["age","gender","emotion"] (if needed).
• Handle response: Returns face IDs, bounding boxes, and attributes.

3. Verify Two Faces (1:1 Matching)

• Call face.verify_face_to_face() with two face IDs (from detection).
• Check the confidence score (e.g., 0.85 means high likelihood of a match).
• Apply a threshold (e.g., reject if < 0.7).

4. Identify a Face (1:N Matching)

• Create a Person Group (face.person_group.create()).
• Add persons (face.person.create()) and enroll faces (face.person.add_face()).
• Train the group (face.person_group.train()).
• Call face.identify() with a detected face ID to find matches.

5. Handle PII & Compliance

• Enable data residency (e.g., store face data in EU-only regions).
• Set faceIdTimeToLive=0 to disable temporary storage (data deleted immediately after processing).
• Implement consent workflows (e.g., "I agree to facial recognition" checkbox).
• Use face.list() to audit stored faces and delete unused data (face.person.delete()).

6. Optimize for Performance & Cost

• Batch detection (up to 10 faces per call for efficiency).
• Use Large Person Groups for enterprise-scale identification (1M+ faces).
• Cache face IDs (they expire after 24h by default).
• Monitor usage in Azure Portal (costs accrue per API call, not per face).

Common Mistakes

Mistake 1: Ignoring Face ID Expiry

• Problem: Face IDs expire after 24h by default, breaking verification/identification workflows.
• Correction: Store face data permanently (e.g., in a Person Group) or re-detect faces before each use.

Mistake 2: Using Face Verification for Identification

• Problem: Candidates confuse 1:1 verification (e.g., selfie vs. ID) with 1:N identification (e.g., matching against a database).
• Correction: Use face.verify_face_to_face() for 1:1 and face.identify() for 1:N.

Mistake 3: Not Training Person Groups

• Problem: Forgetting to train a Person Group after adding faces, leading to failed identification.
• Correction: Always call face.person_group.train() after enrolling new faces.

Mistake 4: Enabling Face Attributes Without Consent

• Problem: Enabling age/gender/emotion detection without user consent violates privacy laws.
• Correction: Disable attributes by default and only enable with explicit consent.

Mistake 5: Using Face API for Non-Face Scenarios

• Problem: Trying to use Face API for object detection (e.g., cars, animals) or OCR.
• Correction: Use Computer Vision API for general object detection or Form Recognizer for OCR.

Certification Exam Insights

1. Service Selection Traps

• Face API vs. Computer Vision API:
• Use Face API for facial recognition (verification/identification).
• Use Computer Vision API for general object detection (e.g., "Is there a person in this image?").
• Face API vs. Custom Vision:
• Face API is pre-trained for faces (no training needed).
• Custom Vision is for custom object detection (e.g., detecting helmets on construction sites).

2. Key Constraints

• Face API has a 4MB image limit (resize large images before sending).
• Person Groups support up to 1,000 faces (use Large Person Groups for 1M+ faces).
• Face IDs expire after 24h unless stored in a Person Group.

3. PII & Compliance Questions

• GDPR/CCPA compliance: Expect questions on data residency, consent, and right to erasure.
• Default settings: Face attributes (age/gender) are disabled by default for privacy.

4. Cost Optimization

• Batch detection (up to 10 faces per call) reduces API calls.
• Free tier: 30,000 transactions/month (but no SLA).

Quick Check Questions

Question 1

A retail store wants to identify VIP customers as they enter. The system must match faces against a database of 50,000 known customers. Which Azure service should they use? ✅ Answer: Face API with a Large Person Group (supports up to 1M faces, ideal for large-scale identification).
❌ Why not? Standard Person Groups max out at 1,000 faces.

Question 2

A bank uses Face API to verify customer identities during mobile onboarding. After 24 hours, the verification fails. What is the most likely cause? ✅ Answer: Face IDs expired (default expiry is 24h; store faces in a Person Group for long-term use).
❌ Why not? Other options (e.g., low confidence score) would fail immediately, not after 24h.

Question 3

A company wants to detect faces in images but does not need facial recognition. They also want to extract text from the images. Which Azure service should they use? ✅ Answer: Computer Vision API (handles face detection + OCR, no facial recognition).
❌ Why not? Face API is overkill for just detection, and it doesn’t do OCR.

Last-Minute Cram Sheet

1. Face API is for facial recognition (detection, verification, identification); Computer Vision API is for general object detection.
2. Face IDs expire after 24h unless stored in a Person Group or Large Person Group.
3. Verification = 1:1 matching (face.verify_face_to_face()); Identification = 1:N matching (face.identify()).
4. Person Groups max at 1,000 faces; Large Person Groups support 1M+ faces.
5. Face attributes (age/gender) are disabled by default for privacy.
6. Train Person Groups after enrolling faces (face.person_group.train()).
7. 4MB image limit for Face API; resize larger images.
8. GDPR/CCPA compliance: Enable data residency, disable temporary storage, and get consent for face attributes.
9. Batch detection: Up to 10 faces per API call for efficiency.
10. ⚠️ Face API does not store images permanently—only face IDs and metadata (unless saved in a Person Group).