All labs
35 hands-on labs, from your first governed call to agent governance in production.
Your first model call in 5 minutes
Learn the building blocks of Azure AI Gateway, create a gateway with a Microsoft Foundry model in the portal, the Azure CLI, Bicep with azd, or Terraform, and make your first model call, all in about five minutes.
How Azure AI Gateway works
A 20-minute visual tour of Azure AI Gateway. Follow one request through the gateway, meet models, aliases, MCP servers, policies, and identities, and pick the learning path that fits your role.
Provision your AI Gateway
Create the shared Azure AI Gateway that your apps and agents will call, from the AI Gateway portal, the Azure CLI, Bicep and azd, or Terraform, then take a guided tour of the portal.
Publish a Foundry model and make your first call
Publish a Microsoft Foundry model deployment through your gateway with its managed identity, create a runtime API key, try the model in the Catalog playground, and call it from curl, PowerShell, Python, TypeScript, and C#.
Publish models from any provider behind stable aliases
Connect Anthropic or another non-Azure provider with a credential that stays in the gateway, publish its models behind callable aliases, call Claude with the native Messages API, and retarget an alias to a different backend model without changing a single client.
Route an alias across models by priority and weight
Put two Foundry models behind one alias, canary a new model version with an 80/20 weighted split, then prefer one model by priority, all without changing the model name that clients send.
Route alias requests with conditions and session affinity
Send the requests that carry a research header to a larger model with a CEL backend condition, then split traffic between two models and keep every conversation on one of them with session affinity, all behind one alias.
Let the gateway choose the model for each request
Put gpt-5-mini and gpt-5.5 behind one alias and let Model Router pick one of them for each prompt. Prove which model served each request, switch the objective between cost and quality, and let callers override it per request.
Fail over to a backup model when a backend fails
Put a deliberately tiny deployment and a healthy backup behind one alias, watch throttling errors reach clients, then add a failure rule that retries on the backup and keeps the throttled backend out for 30 seconds, so every request succeeds.
Stream responses and count every token
Stream Chat Completions and Responses API replies through the gateway, read the server-sent events that your client receives, and prove that a token limit counts streamed tokens even when the client never asks for usage.
Publish embedding models for search and RAG
Publish a Foundry embedding model through the gateway behind its own alias, generate vectors with curl, the OpenAI SDK, and the portal playground, build a mini RAG app on one gateway URL and key, and cap ingestion with a token limit.
Publish Jev decision models for typed answers
Publish TypeSafe's Jev decision model through the gateway with a Custom provider, triage support tickets with typed answers and confidences from curl, PowerShell, the TypeSafe Python SDK, and the portal playground, and cap decision requests per caller with a request rate limit.
Federate MCP tools behind one endpoint
Register a remote MCP server in Azure AI Gateway, add a second backend under its own namespace, and connect curl, VS Code, and Python to one governed MCP endpoint.
Control which tools agents can call
Publish only approved MCP tools with name patterns, block risky tools, take a whole MCP server offline, and cap tool traffic per caller with a request rate limit.
Throttle tokens per caller with token limits
Cap how many tokens each app, agent, or user can consume per minute, hour, or day, and watch the gateway answer with 429 and Retry-After before your backend capacity runs out.
Enforce cost budgets with per-caller overrides
Give every app, agent, and user a dollar budget per hour, day, or month, raise or lower it for specific API keys and Microsoft Entra identities, and watch the gateway answer 403 LlmCostQuotaExceeded once a caller's estimated spend runs out.
Screen prompts and responses with content safety
Add a content safety policy that screens prompts, tool inputs, and responses for hate, self-harm, sexual, and violent content, tune each category's threshold, and see how blocked calls return 400.
Set gateway-wide policy baselines
Apply a baseline of token limit, request rate limit, content safety, and IP filter policies at Gateway scope so every model and MCP server, including the ones you add later, starts with the same guardrails, then layer stricter asset policies on top.
Issue and rotate runtime API keys
Give every app its own runtime API key, keep the value out of your code, rotate it without downtime with the primary and secondary values, revoke a leaked key, and see usage and cost per key in Monitoring.
Go keyless with Microsoft Entra ID and RBAC
Replace shared gateway API keys with Microsoft Entra ID tokens. Grant users, groups, apps, and managed identities an AI Gateway runtime role on all assets, a model alias, or an MCP server, call models from the playground, curl, Python, TypeScript, and C# with no secret in code, and grant management access separately with the control-plane roles.
Call backends with managed identities
Let the gateway reach Microsoft Foundry with a managed identity instead of stored keys. Check the system-assigned identity, then create a user-assigned identity you can share across gateways, grant it Foundry User, attach it, and point a model provider at it from the portal, the Azure CLI, Bicep, or Terraform.
Monitor usage, cost, and traces
Send the gateway's OpenTelemetry metrics, logs, and traces to Application Insights, then use the built-in Monitoring dashboard to see usage, estimated cost per model and API key, policy impact, and gateway-aware traces.
Attribute tokens and cost with OpenTelemetry
Export Microsoft Entra token claims with the gateway's logs, traces, and token metrics, then turn observed tokens into a per-tenant, per-app, and per-key chargeback report with KQL, PromQL, and a small Python script.
Connect agent frameworks to models and tools
Point Microsoft Agent Framework, the OpenAI Agents SDK, and LangChain with LangGraph at the gateway's OpenAI-compatible model endpoint and its MCP endpoint, so every model and tool call an agent makes passes through gateway policies and shows up in Monitoring.
Deploy a Foundry hosted agent behind AI Gateway
Deploy your own Microsoft Agent Framework agent to Foundry Hosted Agents with azd. The agent has no model deployment, no direct tool access, and no keys. It calls its model through AI Gateway with its own Microsoft Entra Agent ID, and reaches the Microsoft Learn MCP server through a Foundry toolbox that signs in with the project's managed identity. Run it locally, invoke it in Foundry, and cap its tokens without changing its code.
Use gateway models in Microsoft Foundry projects
Connect a Microsoft Foundry project to Azure AI Gateway, so that the project's playground, agents, and code call the platform team's models with the project's managed identity, with no model deployments or keys in the project. Then cap the project's tokens with a gateway policy and manage the connection.
Build on your gateway with coding agents
Install the AI Gateway plugin in GitHub Copilot CLI, Claude Code, Cursor, or Gemini CLI, let your coding agent discover the gateway's models and MCP tools and generate working client code, then migrate AI assets from Azure API Management.
Use your gateway's models and tools in GitHub Copilot
Point GitHub Copilot in VS Code, the Copilot CLI, the Copilot SDK, and the GitHub Copilot app at gpt-6.1-sol published by your gateway, then give them Microsoft Learn tools from an MCP server on the same gateway, so every model and tool call uses your key, your limits, and your monitoring.
Use your gateway's models and tools in Claude Code
Point Claude Code at the Claude models that your gateway publishes from Microsoft Foundry, with a token limit and a daily budget for each caller, then give it Microsoft Learn tools from an MCP server on the same gateway, so every model and tool call uses your key, your limits, and your monitoring.
Lock down backend traffic with private networking
Create an Azure AI Gateway with private routing through your virtual network, reach a Foundry backend over a private endpoint with its public access turned off, change the routing of the running gateway, and give the gateway its own inbound private endpoint so you can turn off its public access.
Protect tool calls with Agent 365 threat protection
Have Microsoft Defender and Microsoft Purview evaluate every tool call on an MCP server, before the tool runs, after it returns, or both, and see how a fail-closed denial reaches the caller.
Govern MCP servers with Agent 365
Connect Microsoft Agent 365 to your gateways, find their MCP servers in one inventory, block a single tool or a whole server from the Microsoft 365 admin center, and see exactly what agents get back.
Deploy an enterprise reference architecture
Deploy a production-shaped Azure AI Gateway environment from modular Bicep or Terraform, with a keyless Foundry backend, models and MCP tools behind RBAC, a key per app, layered policies, and monitoring. Then promote it from dev to prod with a GitHub Actions workflow that signs in with OpenID Connect and smoke-tests every deployment.
Govern MCP tool calls with agent hooks
Run your own allow-or-deny rules on every MCP tool call. An agent hooks policy asks an Azure function before the tool runs and again before the result reaches the agent, authenticated with the gateway's managed identity.
Connect Microsoft 365 Copilot to MCP tools as the signed-in user
Let Microsoft 365 Copilot and Copilot Cowork call your MCP tools through Azure AI Gateway as the signed-in user. The gateway checks the user's role, then exchanges the user's token with Microsoft Entra ID on-behalf-of, so the backend sees the user and no secret leaves the gateway.
No labs match those filters
Try fewer filters or a different keyword.