Plan and manage an Azure AI solution
Quotas, Monitoring, and Security for AI Workloads
CoreKeep model and agent workloads within quota, cost, and rate limits; monitor model quality, drift, safety, and grounding as well as search ingestion and relevance; and secure everything with managed identities, keyless authentication, private networking, and least-privilege roles.
Aligned to the AI-103 study guide with skills measured as of April 16, 2026; guide and product behavior verified October 12, 2026.
Why this matters
Production AI failures are usually operational: unexplained 429 errors, a search index that silently stopped refreshing, quality drifting after a model update, or an API key leaked from a config file. AI-103 expects you to pick the metric, alert, identity, or network control that prevents or reveals each of these.
Must Know
- Quota is assigned in tokens per minute (TPM) per subscription, region, model, and deployment type, and each deployment's TPM allocation also sets a proportional requests-per-minute (RPM) limit. Exceeding either returns HTTP 429.
- The TPM check uses an estimate made when the request arrives, based on the prompt plus the max_tokens setting, so an oversized max_tokens can trigger 429 errors even when billed token usage looks low.
- Responses include x-ratelimit-remaining-requests and x-ratelimit-remaining-tokens headers, and 429 responses include a retry delay. The OpenAI SDK retries with backoff by default.
- Provisioned throughput reserves capacity in provisioned throughput units billed per hour. Spillover can route excess traffic from a provisioned deployment to a standard deployment in the same resource.
- Azure Monitor platform metrics for model deployments, such as token usage, request counts, and time to first token, are collected automatically. Resource logs need a diagnostic setting that sends them to a Log Analytics workspace.
- Continuous evaluation samples production traffic for quality and safety evaluators, and scheduled evaluations against the same dataset reveal drift after model or prompt changes.
- Azure AI Search exposes indexer execution status and errors, plus metrics such as search latency and throttled query percentage. Relevance is checked by testing representative queries and evaluating retrieval results.
- Keyless authentication uses Microsoft Entra ID with DefaultAzureCredential, which uses a developer sign-in locally and a managed identity in Azure. Disabling local (key) authentication forces every caller through Entra ID and RBAC.
- Foundry User (formerly Azure AI User) is the least-privilege developer role for building with a project. Foundry Agent Consumer lets a caller use agents without changing them, and Cognitive Services Usages Reader at subscription scope grants quota visibility.
- Disabling public network access with a private endpoint and private DNS restricts inbound traffic. Agent network injection places agent outbound traffic into a delegated subnet in your virtual network, and it must be chosen when the resource is created.
Compare and Distinguish
- Rate limit 429 versus capacity 429: your deployment's TPM or RPM allocation is exhausted versus the shared service is under load, which provisioned capacity avoids.
- Platform metrics versus resource logs: automatic numeric time series versus detailed records that require a diagnostic setting.
- Continuous evaluation versus scheduled evaluation: sampled live traffic versus a fixed dataset run on a schedule to compare over time.
- Control-plane roles versus data-plane roles: managing resources and deployments versus calling models and agents; Contributor can create deployments but cannot call agents.
- Inbound private endpoint versus outbound network injection: who can reach Foundry versus where agent tool traffic leaves from.
Scenario examples
- Scenario: An app receives 429 errors although Azure Monitor shows token usage far below quota. Think: lower max_tokens, smooth request bursts, and read the rate-limit headers.
- Scenario: A search-backed agent began giving stale answers last week. Think: indexer execution history and errors in Azure AI Search, plus alerts on indexer failures.
- Scenario: A web app on App Service must call a Foundry model without storing any secret. Think: a managed identity, DefaultAzureCredential, a Foundry User role assignment, and local authentication disabled.
Exam traps
- Raising quota does not fix 429 errors caused by an inflated max_tokens estimate or bursty RPM.
- Owner and Contributor manage resources but lack the data actions needed to chat with agents; assign Foundry User for that.
- An API key bypasses role checks entirely, so least privilege requires disabling key authentication rather than just adding roles.
- Turning on public access does not remove existing private endpoints, and removing private endpoints does not make a resource public.
- Azure AI Search accepts only API keys by default; role-based access must be enabled on the search service before tokens work.
Key takeaways
- Diagnose 429 errors from headers and request shape before asking for more quota.
- Monitor quality and safety continuously, and monitor retrieval freshness separately from model metrics.
- Default to managed identities, keyless authentication, private networking, and the narrowest role.
How it works
- The service adds each request's estimated maximum token count to a one-minute running total and rejects requests once the TPM limit is reached.
- Private DNS resolves the same Foundry host name to the private endpoint address inside the virtual network, so client code does not change.
Objects and administrative surfaces
- Foundry portal Model quota page and the Operate area of the Foundry control plane.
- Azure Monitor metrics, diagnostic settings, Log Analytics, and Application Insights.
- Azure AI Search indexer status, metrics, and alerts in the Azure portal.
- Access control (IAM), managed identities, private endpoints, and private DNS zones.
When to use it
- Use spillover when a provisioned deployment must absorb occasional peaks without failing requests.
- Use a user-assigned managed identity when several apps share access or access must be granted before the app exists.
Security and governance implications
- Assign Foundry User to both developers and the project managed identity at the Foundry resource scope.
- Search indexers that must reach data through private endpoints need shared private links and a private execution environment.
Troubleshooting signals
- A 401 error from Entra ID authentication often means the resource lacks a custom subdomain endpoint.
- New role assignments can take several minutes to take effect, so an immediate 403 after assignment may resolve on retry.
More detail
- Manage TPM and RPM quota, rate limits, provisioned capacity, spillover, and cost.
- Monitor deployments with Azure Monitor and evaluations for quality, drift, safety, and groundedness.
- Monitor search ingestion, index health, latency, throttling, and relevance.
- Configure managed identity, keyless access, private endpoints, network injection, and Foundry roles.
Ready for the quiz?
- Why can max_tokens cause throttling without increasing billed tokens?
- Which data source tells you that a search indexer has stopped processing new documents?
- Which role lets a developer build and test agents without full resource ownership?
- What does disabling local authentication change for callers that use API keys?
Related objectives
- D1.3.S1 — Manage quotas, scaling, rate limits, and cost footprints for model and agent workloads
- D1.3.S2 — Monitor model performance, drift, safety events, and grounding quality
- D1.3.S3 — Monitor data ingestion quality, search index health, and relevance performance
- D1.3.S4 — Configure security, including managed identity, private networking, keyless credentials, and role policies