Azure AI Gateway Labs
Documentation Open portal

What's new in the labs

Every Azure AI Gateway release, and where to practice it. The labs follow the product release notes: each entry lists the new product features and the lab that covers each one.

What changed in the Azure AI Gateway labs for each product release, and where to practice every new feature. Newest first. 🆕 marks a new lab, ✏️ an updated lab, ✅ a lab that already covered the feature, and ➖ a feature that no lab covers yet.

November 18, 2026

Azure AI Gateway is generally available

Azure AI Gateway is generally available. Every lab now uses the generally available portal and management API version 2026-11-01, and every track was retested on it. Read the product release notes.

In the productPractice it in the labsStatus
Create a gateway, import a model, and make your first call in the generally available portalQuick start · Your first model call in 5 minutes and 01 · Provision your AI Gateway✏️ Updated
Manage gateways and their assets with the management API 2026-11-01 in REST and BicepEvery lab's REST and Bicep + azd tracks, starting with 02 · Review the templates✏️ Updated
Manage gateways and their assets with the az aigateway extension 1.0.0b34, which calls 2026-11-01 and prints the generally available field namesEvery lab's Azure CLI track, starting with Install the aigateway extension, and 08 · Set the policy, which writes content safety with az aigateway model update --policies✏️ Updated
Manage gateways and their assets with Terraform and the AzAPI provider at 2026-11-01A new Terraform track in the quick start and 28 labs, starting with Deploy a lab with Terraform and 01 · Provision your AI Gateway✏️ Updated
Providers set modelProviderType and a displayName, and a model's supportedEndpoints decide its API format02 · Publish a Foundry model and make your first call and 03 · Publish models from any provider behind stable aliases✏️ Updated
Tool server backends set mode (failClosed or failOpen), and tool lists use blockList04 · Federate MCP tools behind one endpoint and 05 · Control which tools agents can call✏️ Updated
Request rate limits set a count and a period05 · Control which tools agents can call and 09 · Review the policy file✏️ Updated
Cost limit overrides for Microsoft Entra identities list claim name and value pairs07 · Give a Microsoft Entra identity its own budget✏️ Updated
Token limits and content safety policies on the generally available API06 · Throttle tokens per caller with token limits and 08 · Screen prompts and responses with content safety✏️ Updated
Deploy the enterprise reference architecture with the generally available templates21 · Review the templates and 21 · Apply the gateway baseline✏️ Updated
Current agent SDKs and the coding-agent plugin15 · Connect agent frameworks to models and tools, 16 · Deploy a Foundry hosted agent behind AI Gateway, and 17 · Build on your gateway with coding agents✏️ Updated
Import Foundry deployments filtered by resource group24 · Import the deployment and 25 · Import the Claude deployments✏️ Updated
Back a model provider with a system- or user-assigned identity (authentication.managedIdentity), or an API-key header12 · Point the model provider at the identity and 12 · Use stored credentials when a backend can't take a managed identity✏️ Updated
Telemetry exporters set telemetryExporterType, {url} endpoints, and a top-level authentication block13 · Create the telemetry exporter and 14 · Choose the claims to export✏️ Updated
Configure telemetry in the portal on a managed Log Analytics workspace13 · Start the setup✏️ Updated
Turn off a gateway's public network access: its public endpoint returns 40318 · Turn off the gateway's public network access✏️ Updated
Request inbound private endpoints, find the gateway's private DNS zone, and reach the gateway on a private IP18 · Inbound private endpoints and 18 · Reach the gateway through an inbound private endpoint✏️ Updated
Protect tool calls with tool threat protection on the generally available tool server shape19 · Add tool threat protection✏️ Updated
Allow exact tool names by pattern, block tools, and suspend MCP server access20 · Allow the exact tool names and 20 · Block the whole server✏️ Updated

November 12, 2026

Model failover and retry rules

Keep an alias available when a backend deployment is throttled or fails: retry the same backend, switch the request to the next eligible model, and temporarily exclude the failing backend from later requests. Read the product release notes.

In the productPractice it in the labsStatus
Ordered failure rules on an alias, with a CEL condition over the backend response29 · Turn on failover🆕 New lab
Switch a failed request to the next backend model by priority, and prove it with x-ms-served-model29 · Watch the failover🆕 New lab
Temporarily exclude a failing backend (excludeFor)29 · Check that the primary comes back🆕 New lab
Same-backend retries and the per-request retry limit (retryAttempts, maxRetryAttempts)29 · How it works🆕 New lab
Failure handling in the Create alias wizard and the alias editor29 · Add a failure rule🆕 New lab
Priority alone passes backend errors to the client; add failure rules to fail over22 · How it works✏️ Updated
Routing attributes and metrics in telemetry (azure.ai_gateway.routing.*)13 · Read the routing attributes✏️ Updated

November 9, 2026

Jev decisions and per-request model selection

Publish TypeSafe Jev decision models behind an alias, and let callers override auto model selection on individual requests. Read the product release notes.

In the productPractice it in the labsStatus
Publish TypeSafe Jev models behind a gateway alias with a Custom provider31 · Publish Jev behind an alias🆕 New lab
Evaluate yes/no questions, categorical choices, and scoring rubrics with native System One requests over uncompressed JSON31 · Send a decision request🆕 New lab
Limit decision requests per caller31 · Limit decision requests per caller🆕 New lab
Allow callers to override the objective or narrow the model pool for one request (preview)28 · Let callers choose per request🆕 New lab
Filter auto model selection for function-tool or request-parameter compatibilityExplained in 28 · Go further➖ Not covered
Priority and weighted routing, and decision telemetry, for Jev modelsExplained in 31 · Go further and 13 · Read the routing attributes➖ Not covered

November 6, 2026

Auto model selection, session affinity, Agent Hooks, and on-behalf-of authentication

Combine prompt-aware model selection with gateway routing, and add delegated authentication and custom governance to MCP backends. Read the product release notes.

In the productPractice it in the labsStatus
Choose a model for each prompt with auto model selection and a Balanced, Cost, or Quality objective (preview)28 · Let the gateway choose the model for each request🆕 New lab
See which model served a request with x-ms-served-model28 · Send a simple and a hard prompt🆕 New lab
Route an alias's requests by request headers or the caller with CEL backend conditions27 · Route research requests to the larger model🆕 New lab
Keep a conversation on one backend model with session affinity27 · Keep each session on one backend🆕 New lab
Write and check CEL expressions in the portal27 · Create the chat alias with a condition🆕 New lab
Understand the routing order: conditions, priority, auto selection, session affinity, and weights22 · How it works and 27 · Go further✏️ Updated
Stateful API continuations stay pinned to their owning backend23 · How it works✏️ Updated
Ask your decision service before an MCP tool runs, and enforce its allow or deny verdict (Agent Hooks, preview)30 · Call a blocked tool🆕 New lab
Screen tool results before they reach the agent with an after-tool-call hook30 · Get a result that contains a secret🆕 New lab
Choose fail closed or fail open when the decision service is unavailable30 · See the policy fail closed🆕 New lab
Grant management, read, and runtime access separately with the control-plane roles11 · Grant control-plane access✏️ Updated
Assign runtime roles on all assets, a model alias, or an MCP server11 · Grant a runtime role✏️ Updated
Mark failed model streams and MCP tool calls as Error in telemetry, even when the response is 20013 · Find failed operations✏️ Updated
Count failed operations per caller in a chargeback report14 · Build a chargeback report✏️ Updated
Call MCP backends as the signed-in user with on-behalf-of token exchange33 · Connect Microsoft 365 Copilot to MCP tools as the signed-in user, explained in 12 · Call MCP backends as the signed-in user🆕 New lab
Edit virtual network settings after gateway creation: add, change, or remove the integration subnet18 · Remove and add back private routing and 01 · Review the template✏️ Updated
Send runtime keys as Authorization: Bearer <key>10 · Send the key in either header and 02 · How it works✏️ Updated
Fixed: stream Anthropic Messages without stream_options failures25 · Ask a quick question✅ Covered
The new routing options, Agent Hooks, control-plane roles, and failure telemetry in the big picture00 · How Azure AI Gateway works✏️ Updated

October 28, 2026

Embedding models

Import embedding models, publish stable aliases, and generate vectors in the portal. Read the product release notes.

In the productPractice it in the labsStatus
Import embedding deployments from Foundry and publish them behind stable aliases26 · Publish embedding models for search and RAG🆕 New lab
Generate vectors with text batches and backend-supported dimensions26 · Generate embeddings🆕 New lab
Test embeddings in the Catalog playground and Continue in code26 · Test in the playground🆕 New lab
Apply token limits and usage telemetry to embedding models26 · Cap ingestion with a token limit🆕 New lab
Route an embeddings alias across deployments by priority and weightExplained in 26 · Go further, practiced with chat models in 22 · Route an alias across models by priority and weight➖ Not covered

October 16, 2026

Priority and weighted model routing, MCP 2026-07-28

Keep a stable alias while you prefer purchased capacity, balance traffic across deployments, or test a new model version, and manage MCP servers that use the 2026-07-28 MCP specification. Read the product release notes.

In the productPractice it in the labsStatus
Route an alias's traffic across backend models by priority and weight22 · Route an alias across models by priority and weight🆕 New lab
Canary a new model version or run an A/B test with a controlled split22 · Split traffic 80/20 between two models🆕 New lab
Prefer purchased capacity with priorities22 · Prefer one model by priority🆕 New lab
Create and edit aliases with the Create alias wizard03 · Publish models from any provider behind stable aliases✏️ Updated
Choose the MCP protocol versions a server supports, including 2026-07-2804 · Choose supported MCP versions✏️ Updated
See supported MCP versions on the server page and in Catalog04 · Check the negotiated version✏️ Updated

October 8, 2026

Agent 365 integration and tool threat protection

Discover and govern AI Gateway MCP servers and tools from Microsoft Agent 365, and protect tool calls with Microsoft Defender and Microsoft Purview. Read the product release notes.

In the productPractice it in the labsStatus
Discover AI Gateway MCP servers in the Agent 365 inventory20 · Connect Agent 365 to your gateways✅ Covered
Block or unblock MCP servers and individual tools from Agent 36520 · Block a tool and 20 · Block the whole server✅ Covered
Evaluate tool calls before and after invocation with Defender and Purview19 · Add tool threat protection and 19 · Understand verdicts and failures✅ Covered

October 1, 2026

Cost limit overrides for Microsoft Entra identities

Set different cost limit amounts for Microsoft Entra callers by identity or by an exact combination of validated token claims. Read the product release notes.

In the productPractice it in the labsStatus
A higher or lower budget for a selected identity or matching token claims07 · Give a Microsoft Entra identity its own budget✅ Covered
Separate budgets per caller by tenant ID and object ID07 · Enforce cost budgets with per-caller overrides✅ Covered
The first matching override wins07 · How it works✅ Covered

September 24, 2026

Model aliases, Entra access, and Chat Completions streaming usage

Aliases become the stable model contract, with Microsoft Entra authentication, gateway-scoped policies, and complete usage for streamed Chat Completions responses. Read the product release notes.

In the productPractice it in the labsStatus
Callable aliases, several aliases for one backend model, and retargeting03 · Publish models from any provider behind stable aliases✅ Covered
Grant model and MCP access to Microsoft Entra users, groups, apps, and managed identities11 · Go keyless with Microsoft Entra ID and RBAC✅ Covered
Export selected JWT claims to OTLP logs, traces, and metrics14 · Choose the claims to export✅ Covered
Apply one policy to every current and future asset, including a gateway-wide cost limit09 · Set gateway-wide policy baselines✅ Covered
Usage for streamed Chat Completions responses without include_usage23 · Stream a burst without asking for usage🆕 New lab

September 10, 2026

Per-key budget overrides and MCP tool filtering and blocking

Assign higher or lower model budgets to selected API keys, and control which MCP tools agents can discover and invoke. Read the product release notes.

In the productPractice it in the labsStatus
A different budget for selected API keys07 · Give an API key a bigger budget✅ Covered
Expose all, selected, or pattern-matched tools05 · Publish only approved tools✅ Covered
Block individual tools or a whole MCP server05 · Take the whole server offline✅ Covered
Redesigned Home and first-run gateway setup01 · Provision your AI Gateway✅ Covered

August 28, 2026

OpenTelemetry observability and model budgets

Export logs, traces, token metrics, and estimated cost metrics through OTLP, and enforce model spend with cost limits. Read the product release notes.

In the productPractice it in the labsStatus
The built-in Application Insights dashboard and request traces13 · Monitor usage, cost, and traces✅ Covered
OTLP export of logs, traces, and token and cost metrics14 · Attribute tokens and cost with OpenTelemetry✅ Covered
Cost limits with hourly to yearly budget windows07 · Enforce cost budgets with per-caller overrides✅ Covered
Import Microsoft Foundry models and onboard custom providers02 · Publish a Foundry model and 03 · Add Anthropic as a custom provider✅ Covered

How these notes work

Each section matches one entry in the product release notes, with the same date and title, and maps every product change to the lab where you can practice it.

This file, labs/RELEASE-NOTES.md, is the source of truth: the labs site renders it at /release-notes/. To check for product releases that the notes don't cover yet, run npm run releases in labs/. See Authoring.