No errors match your filter. Try an error code such as 429, or a word from the message.
Setup and deployment
Setup guide: The one-time tasks that every lab builds on
| Symptom | Likely cause | Fix |
|---|---|---|
'aigateway' is misspelled or not recognized by the system | The aigateway extension isn't installed | Install the extension, then run az aigateway version. |
MissingSubscriptionRegistration, or The subscription is not registered to use namespace | A resource provider isn't registered in the subscription | Register the resource providers, or ask your subscription administrator. |
LocationNotAvailableForResourceType | The location isn't a region where AI Gateway is available | Pick a region from Find the available regions. |
RequestDisallowedByAzure: The selected region is currently not accepting new customers | The region has AI Gateway, but your subscription can't create new resources there | Pick another region from the list. |
InvalidResourceGroupLocation from azd up | The resource group rg-<environment name> already exists in another region | Run azd env new with a new environment name, or delete the old resource group. |
| Your subscription is disabled in the portal's subscription picker | You lack Azure AI Gateway write access on that subscription | Ask for Contributor on the subscription, or pick another subscription. |
terraform apply or azd up waits on the gateway for 20 minutes or more, with Still creating... on azapi_resource.gateway | A new gateway stays Activating until it's ready, which takes from about a minute to more than half an hour, depending on the region | Wait: the labs' Terraform configurations wait up to an hour for the gateway, and azd up waits until it's ready. To check the gateway, open it in the portal and look at Provisioning. |
context deadline exceeded on azapi_resource.gateway, or Resource already exists when you run terraform apply again | The gateway took longer than Terraform waited, or you stopped terraform apply, so the gateway exists in Azure but not in the Terraform state | Import the gateway at the address that the error shows after with: terraform import '<address>' '<gateway resource ID>?api-version=2026-11-01'. For example, the address in lab 01 is module.gateway.azapi_resource.gateway. Wait until Provisioning shows Succeeded on the gateway's Overview page, then run terraform apply again. |
ServiceLocked (409 Conflict) on azapi_resource.gateway | The gateway is still activating, and Azure doesn't accept changes until it finishes | Wait until Provisioning shows Succeeded on the gateway's Overview page, then run terraform apply again. |
zsh: command not found: #, or a pasted block stops after a comment | zsh on macOS reads pasted # lines as commands | Turn on interactive_comments, as in Choose where to run commands, and paste the block again. |
jq: error: syntax error, unexpected INVALID_CHARACTER, and the pasted command shows \{ or \} after a URL | The block was pasted while a command still ran, and oh-my-zsh's URL quoting escaped the characters after the URL | Wait for the prompt, then paste the block again. |
Pasted text shows codes such as <0001f9e0> | The snippet is an older copy with an emoji that zsh can't draw. The command still runs. | Reload the lab page to copy the current snippets. |
| The portal shows Access required for a gateway that you own | The portal is signed in to a different directory (tenant) than the gateway's subscription | Switch the portal to the directory that holds your subscription, then open the gateway again. |
Quick start
Quick start: Your first model call in 5 minutes
| Symptom | Likely cause | Fix |
|---|---|---|
| A subscription is disabled in Create AI gatewayStep Create the gateway | You lack Azure AI Gateway write access on it | Pick another subscription, or ask for Contributor on it. |
AuthorizationFailed on Microsoft.Authorization/roleAssignmentsSteps Import the model from FoundryPublish the modelDeploy with azd upDeploy with terraform apply | You can create resources but not role assignments | Ask for Owner, or Role Based Access Control Administrator, on the resource group. |
LocationNotAvailableForResourceTypeSteps Start creating the gatewayDeploy with azd upDeploy with terraform apply | AI Gateway isn't available in the region | Pick a region from Find the available regions. |
RequestDisallowedByAzure: The selected region is currently not accepting new customersSteps Create the gatewayStart creating the gatewayDeploy with azd upDeploy with terraform apply | The region has AI Gateway, but your subscription can't create new resources there | Pick another region from the list. |
InvalidResourceProperties: The specified SKU 'GlobalStandard' for model 'gpt-5-mini 2025-08-07' is not supported in this regionSteps Deploy the model in FoundryDeploy with azd up | gpt-5-mini isn't available in the region | Pick a region where the model check in Find the available regions prints GlobalStandard. |
InsufficientQuota on the model deploymentSteps Deploy the model in FoundryDeploy with azd upDeploy with terraform apply | The subscription has no gpt-5-mini capacity left in the region | Start again in another region: set LOCATION in the Azure CLI track, run azd env new with a new name and another --location in the Bicep + azd track, or run terraform destroy -auto-approve and set TF_VAR_location in the Terraform track. Or free capacity in Foundry. |
SpecialFeatureOrQuotaIdRequired: The current subscription does not have access to this model on the Foundry deploymentStep Deploy with terraform apply | gpt-5-mini isn't available in TF_VAR_location, for example westus2 | Run terraform destroy -auto-approve, set TF_VAR_location to a region where the model check in Find the available regions prints GlobalStandard, then plan and apply again. |
InvalidResourceGroupLocation from azd upStep Deploy with azd up | The resource group rg-<environment name> already exists in another region | Run azd env new with a new environment name, or delete the old resource group. |
401, 403, or 404 from the gateway in the first minuteStep Make your first call | The new key, the new alias, or the gateway's role on the Foundry account is still taking effect | Wait 15 to 60 seconds and call again. |
403 from the backend that doesn't go awayStep Make your first call | The gateway's managed identity has no role on the Foundry account | Run the import in the Azure CLI track's Publish the model step again, which grants the role, or assign the gateway Foundry User on the account. Then wait a few minutes. |
401: Missing or invalid Entra ID token, or missing Data.All scopeStep Make your first call | The key is missing, wrong, in the wrong header, or was created less than a minute ago. The message mentions Entra ID even when you use a key | Send it as Authorization: Bearer <key>. In the portal, copy it again from AccessAPI keys. A new key can take a minute to work, so wait a minute and retry. |
The playground shows You don't have permission to invoke this model with Entra ID and you can't type a promptStep Make your first call | Method is Microsoft Entra ID, and you have no AI Gateway role on the gateway | Set Method to API key and choose app-first-call. To call with Entra ID instead, grant yourself AI Gateway Models User in AccessEntra ID, as Lab 11 shows. |
400 that mentions authenticationStep Make your first call | The request has both an api-key and an Authorization header | Send the key only as Authorization: Bearer <key>. |
The subscription is not registered to use namespaceSteps Start creating the gatewayDeploy the model in FoundryDeploy with azd upDeploy with terraform apply | A resource provider isn't registered yet | Register the resource providers, then try again. |
404 with Resource not found from the gatewayStep Make your first call (Azure CLI, Bicep + azd) | The alias is missing, or the URL path is wrong | Run the alias command in Publish the model: the import adds models but no aliases. Call <runtime URL>/default/models/openai/v1/chat/completions. |
Provisioning shows Created or InProgress for several minutesStep Create the gateway | The gateway is still being created | Wait until the status shows Succeeded, then continue. |
The deployment's checkbox is disabled, with This model provider already exists in your gateway. No new models are available.Step Import the model from Foundry | The account is already imported | Close the page and create the alias from ModelsAliases instead. |
400: A model alias is required in the request body.Step Make your first call | The request has no model field | Send the alias in model, for example "model": "gpt-5-mini". |
400: The displayName property is required. on a modelProviders or models resourceStep Deploy with azd up | A template written for the preview API, deployed at 2026-11-01 | Add displayName to the provider and the model, as infra/resources.bicep does. |
Lab 00: How AI Gateway works
Lab 00: How Azure AI Gateway works
| Symptom | Likely cause | Fix |
|---|---|---|
| A model call says the model can't be foundStep Browse models and aliases | The model field doesn't match an alias | Copy the alias from Models, or from the code sample in Catalog |
| An Anthropic call fails on the OpenAI endpointStep Browse models and aliases | The gateway doesn't translate between protocols | Call /default/models/anthropic/v1/messages with anthropic-version: 2023-06-01 |
| You can't create a gateway in your regionStep Open your gateway | AI Gateway isn't available there yet | Pick a region from Find the available regions |
| Monitoring is emptyStep Open Monitoring | No telemetry destination, or no traffic yet | Connect Application Insights as in Lab 13, then send a few requests |
| The wizard's last step is Details, and nothing saves when you leaveStep Start a policy | Cancel discards the policy | That's expected in this tour. Lab 06 saves a policy. |
Lab 01: Provision a gateway
Lab 01: Provision your AI Gateway
| Symptom | Likely cause | Fix |
|---|---|---|
| Your subscription is disabled in the portal's subscription pickerStep Enter the gateway details | You lack Azure AI Gateway write access on that subscription | Ask for Contributor on the subscription, or pick another subscription. |
LocationNotAvailableForResourceTypeSteps Create the AI gatewayDeploy with azd upDeploy with terraform apply | $LOCATION or the template's location isn't a region where AI Gateway is available | Pick a region from Find the available regions. The template defaults to eastus2. |
RequestDisallowedByAzure: The selected region is currently not accepting new customersSteps Watch it provisionCreate the AI gatewayDeploy with azd upDeploy with terraform apply | The region has AI Gateway, but your subscription can't create new resources there | Pick another region from the list. |
| The name is rejectedSteps Enter the gateway detailsCreate the AI gateway | Gateway names are globally unique and limited to 24 characters: letters, digits, and hyphens, starting with a letter | Pick another name that follows the rules. |
AIGatewaySubscriptionGatewayLimitExceeded: …has reached the maximum of 5 AI Gateway resources allowed in this region.Steps Watch it provisionCreate the AI gatewayDeploy with azd upDeploy with terraform apply | The subscription already has five gateways in $LOCATION | Delete a gateway you no longer need, or create this one in another available region. |
InvalidSkuName: The supported AI Gateway SKU name is 'Standard'.Step Deploy with azd up | Your template or command sets another SKU, such as Consumption, Premium, or AIGateway from older samples | Set sku: { name: 'Standard' }, as infra/gateway.bicep does, or omit --sku in az aigateway create. |
MissingSubscriptionRegistrationSteps Create the AI gatewayDeploy with azd upDeploy with terraform apply | A resource provider isn't registered in the subscription | Register the resource providers, or ask your subscription administrator. |
Could not resolve host when you call the URLStep Call the runtime URL | Provisioning or DNS isn't finished | Run az aigateway wait -g "$RG" -n "$GATEWAY" --created and retry. |
'aigateway' is misspelled or not recognized by the systemSteps Create the AI gatewayCall the runtime URL | The aigateway extension isn't installed | Install the aigateway extension, then run az aigateway version. |
403: Public network access is disabled for this resource.Step Call the runtime URL | publicNetworkAccess is Disabled, which blocks calls from the internet | Call from the virtual network through a private endpoint, or set public network access back to Enabled in Networking. |
The status stays Created or InProgressStep Watch it provision | Provisioning takes a few minutes | Wait for Succeeded. With the CLI, run az aigateway wait -g "$RG" -n "$GATEWAY" --created. |
Lab 02: First model call
Lab 02: Publish a Foundry model and make your first call
| Symptom | Likely cause | Fix |
|---|---|---|
401 from the gatewaySteps Call Chat CompletionsRun the sampleGet a 200 with the key | The key is missing, misspelled, or deleted | Send an active runtime key as Authorization: Bearer <key> or as the api-key header (not both), or as the SDK API key. Create a new key if you lost the value. |
400 with Specify either an API key or an Authorization header, not bothStep Run the sample | The client sends an api-key header and an Authorization header, such as an SDK placeholder key | Pass the runtime key as the SDK API key and remove the api-key default header. |
403 mentioning permissions on the Foundry accountSteps Try the model in CatalogCall Chat CompletionsRun the sampleGet a 200 with the key | The gateway's identity lacks Foundry User, or the assignment hasn't propagated yet | Wait a few minutes. If it persists, assign the role as shown in the portal track. |
404 with model_not_foundSteps Call Chat CompletionsRun the sampleGet a 200 with the key | model doesn't match an alias exactly | Copy the alias from the model's Aliases tab or from Catalog. |
400 unsupported_parameter for max_tokens, or unsupported_value for temperatureSteps Call Chat CompletionsRun the sample | gpt-5-mini is a reasoning model | Send max_completion_tokens instead of max_tokens. Leave temperature out: the model accepts only the default, 1. |
200 with empty content and finish_reason lengthSteps Call Chat CompletionsRun the sample | Reasoning used the whole max_completion_tokens budget | Raise the budget, or send reasoning_effort: "none" for short, direct replies. |
| The Responses API call fails while Chat Completions worksSteps Call the Responses APIRun both calls with a scriptRun the sample | The model doesn't advertise the Responses interface | Check the supported interfaces on the model details page and call only those. |
| The import wizard doesn't list your accountStep Import the deployment from Foundry | It's already imported into this gateway, or you can't read it in that subscription | Check Models for an existing provider, or ask for read access to the account. |
AuthorizationFailed on roleAssignments during deploymentSteps Grant the gateway access to the accountDeploy with azd up | You can't create role assignments on the Foundry account | Ask for Role Based Access Control Administrator on the account, or have an owner grant the role. |
You must install or update .NET to run this application.Step Run the sample (Code) | The sample targets .NET 8, and only a later runtime is installed | Install the .NET 8 runtime, or run with DOTNET_ROLL_FORWARD=Major. |
400: The displayName property is required.Steps Register the deploymentDeploy with azd up | The provider or model body has no displayName at 2026-11-01 | Add displayName, as infra/publish-model.bicep does. |
400: A model alias is required in the request body.Step Call Chat Completions | The request has no model field | Send the alias in model. |
The provider check prints "kind": null, and endpoint is an object with urlStep Create the Foundry provider | The query is written for the aigateway extension 1.0.0b33 or earlier. From 1.0.0b34, the extension prints the names of API version 2026-11-01 | Query properties.modelProviderType and properties.foundry.endpoint.url, as the step does. |
Lab 03: Multi-provider aliases
Lab 03: Publish models from any provider behind stable aliases
| Symptom | Likely cause | Fix |
|---|---|---|
| The wizard discovers no models, or times outStep Choose the models and their aliases | The base endpoint URL includes /v1, the header name is wrong, or the provider doesn't serve /v1/models | Use https://api.anthropic.com and x-api-key. If the provider can't list models, add them with Individual model. |
401 from the gatewayStep Call both aliases | The Authorization header is missing or holds the Anthropic key instead of a gateway runtime key | Send a runtime key from AccessAPI keys as Authorization: Bearer <key>. |
| An authentication error from Anthropic in the response bodyStep Call both aliases | The stored provider key is wrong or expired, or Include Bearer is selected for Anthropic | Update the provider's API key and clear Authentication scheme. |
400 that mentions anthropic-versionStep Call both aliases | The request doesn't send the anthropic-version header | Add anthropic-version: 2023-06-01. |
404 or a "model not found" errorStep Call both aliases | The model value doesn't exactly match an alias, or you called the OpenAI route for an Anthropic model | Copy the alias from Models, and call /default/models/anthropic/v1/messages. |
404 with an empty body from both Anthropic aliases, while gpt-5-mini worksStep Call both aliases | A service issue at the time of writing: the gateway adds its /anthropic route segment to the provider's endpoint path, so Anthropic gets /anthropic/v1/messages and answers 404 | Your setup is right, and no endpoint value works around it. Finish the other steps with gpt-5-mini, and check the AI Gateway changelog for the fix. |
Error: No value for required variable for anthropic_api_keySteps Deploy with terraform applyChange the backend model with a variableDelete everything Terraform created | The key is an ephemeral variable that Terraform never stores, so every command needs it, and a new terminal doesn't have it | Read the key into TF_VAR_anthropic_api_key again with the track's hidden prompt. |
Unsupported Terraform Core versionStep Deploy with terraform apply | Terraform is older than 1.11, which the write-only sensitive_body argument needs | Install Terraform 1.11 or later. |
support_chat_model takes an empty string, fast_model, or smart_model.Step Change the backend model with a variable | The -var names a model that the Anthropic provider doesn't publish | Pass $FAST_MODEL or $SMART_MODEL, or an empty string for the default. |
| "Callable aliases must be unique"Steps Choose the models and their aliasesAdd the batch-chat alias | Another alias in the gateway already uses that name | Pick a different alias, or retarget the existing one. |
Lab 04: Federate MCP tools
Lab 04: Federate MCP tools behind one endpoint
| Symptom | Likely cause | Fix |
|---|---|---|
401 from the MCP endpointSteps Add the server to VS CodeCall the tools from PythonList the published toolsSend a raw JSON-RPC request | The key is missing, revoked, or sent in the wrong header | Send a runtime key from AccessAPI keys as Authorization: Bearer <key>. |
404 from the MCP endpointSteps Add the server to VS CodeCall the tools from PythonList the published toolsSend a raw JSON-RPC request | Wrong server name in the URL, or the server is still provisioning | A new server can answer 404 for about 20 seconds, so wait and retry. If it persists, list the servers with az aigateway mcp list -g "$RG" --gateway-name "$GATEWAY" -o table and use the exact name in /default/toolservers/<server-name>/mcp. |
406 or an error about the Accept headerStep Send a raw JSON-RPC request | The client doesn't accept both JSON and SSE | Send Accept: application/json, text/event-stream, or use an MCP SDK. |
tools/list returns an empty tools arraySteps Call the tools from PythonList the published toolsSend a raw JSON-RPC request | The registration is reachable but no backend returned tools | Check each backend's URL with az aigateway mcp source list -g "$RG" --gateway-name "$GATEWAY" -n <server-name> -o table, and check its authentication. |
| The whole server fails when one backend is downSteps Call the tools from PythonList the published toolsCall a tool through the gateway | That backend is required: Required in the portal, mode: 'failClosed' in Bicep | Turn Required off for backends that are nice-to-have (--required false in the CLI, mode: 'failOpen' in Bicep), or fix the backend. |
| Backends or policies disappeared after a Bicep deploymentSteps Add the second namespacePin the versions | A Bicep deployment sends a PUT, which replaces an existing registration with the same name | Change existing servers with az aigateway mcp update or az aigateway mcp source add and remove, as in the CLI track, or deploy to a new name. |
mcp create fails with "This MCP registration already exists; use mcp update."Step Create the registration | A server with that name already exists | Change it with az aigateway mcp update or the mcp source commands, or choose another name. |
mcp source remove fails with "Cannot remove the last source."Step Remove the second source or the server | An MCP server needs at least one backend | Keep at least one backend, or delete the server with az aigateway mcp delete. |
400 with Unsupported MCP-Protocol-VersionSteps Add the server to VS CodeCall the tools from Python | The client sent an MCP-Protocol-Version header that isn't in Supported MCP versions | Add the version to the list if every backend supports it, or let the client negotiate a listed version. |
404 with "Federation tools/list failed: required endpoint '…' returned error: Backend selected protocol version '…' instead of '…'"Steps Add the server to VS CodeCall the tools from PythonList the published tools | Supported MCP versions includes a version that this backend doesn't support, and the client negotiated it | Keep only versions that every backend supports. In the portal, edit and save each backend to recheck it. |
400 with JSON-RPC error -32603, "The request cannot be completed using the requested protocol version."Steps Add the server to VS CodeCall the tools from Python | A 2026-07-28 request reached a backend that doesn't support 2026-07-28 | Remove 2026-07-28 from Supported MCP versions until every backend supports it. |
400 with JSON-RPC error -32602, "Required MCP request metadata is missing or malformed."Steps List the published toolsSend a raw JSON-RPC request | A 2026-07-28 request lacks the Mcp-Method or Mcp-Name header, or params._meta | Use an MCP SDK that implements 2026-07-28 instead of hand-written requests. |
| The portal shows "Backend isn't compatible with all configured MCP versions."Step Pin the version | The backend doesn't report support for every selected version | Return to Basic info and change Supported MCP versions, or update the backend. |
Creating a server fails with 502 Bad Gateway and an HTML page, or a Bicep deployment keeps runningSteps Publish all toolsCreate the registrationDeploy with azd up | Another MCP server in the gateway already uses the same display name or namespace, or the URL isn't an MCP endpoint | Choose a unique display name and namespace, for example with the Bicep displayName parameter, and check that the URL answers MCP requests. |
| After Next on Backends, the portal returns to MCP servers instead of opening ToolsStep Publish all tools | The portal saves the server with Allow all tools at this point | Nothing to fix: the server publishes every tool. Open it to copy the endpoint |
| The Description reads "Federated MCP server providing tools from: learn"Step Copy the endpoint | The portal replaced your description when it saved the backends | Select Edit, enter the description again, and select Save |
| Save shows "Supported MCP versions changed. Edit and save each external MCP backend before saving this server."Step Pin the version | Every backend must be rechecked against the new version list | Edit each MCP backend, select Save on it, then save the server |
az aigateway mcp show prints required and mcp.url instead of mode and mcp.endpoint.url, or the step's --query prints only a Namespace columnSteps Create the registrationConfirm the registrationAdd the second namespace | The aigateway extension is older than 1.0.0b34, which calls the preview management API | Run the installer again, then check extensionVersion with az aigateway version. |
| A backend that should be required fails open when its server is downStep Deploy with azd up | The template sets required: true, which the GA API ignores without an error | Set mode: 'failClosed' on the endpoint. Use mode: 'failOpen' for an optional backend |
list-tools fails with 503 and {"error":"App port is not ready"}Steps List the published toolsCall a tool through the gateway | The MCP server's runtime is still starting, usually in the first minutes after you create it | Wait a few seconds and run the script again. |
Lab 05: Tool allow and block lists
Lab 05: Control which tools agents can call
| Symptom | Likely cause | Fix |
|---|---|---|
A tool you blocked still appears in tools/listSteps Check the catalogCall a blocked tool | Expected. Blocked tools remain visible, but the gateway blocks their execution | Call the tool to confirm that it's blocked. To hide a tool, leave it out of the allow list instead. |
tools/list returns no toolsStep Check the catalog | The allow list matches nothing | Check the patterns against the exact tool names. |
initialize returns AccessBlockedByGatewaySteps Turn the server back onCall a blocked toolRun the burst | The server access is Blocked | Set MCP server access to Active. With the Azure CLI, run the az rest command of Turn the server back on. With azd, run azd env set MCP_SERVER_ACCESS active and azd provision. |
| An allowed tool is blockedStep Check the catalog | A blocked pattern also matches it. Blocked patterns take precedence | Make the blocked pattern narrower, for example *code_sample* instead of *sample*. |
429 on the first request of a runSteps Call a blocked toolRun the burst | The previous step used up the window, or a Gateway scope rate limit also applies | Wait for Retry-After, and review Policies for every limit that covers the MCP server. |
Never 429Step Run the burst | The policy is on another asset, it was saved moments ago, or the burst spans two windows | Check the policy's scope, wait a minute after you save it, and run a longer burst, for example ./infra/trigger-request-rate.sh learn 12. |
Other policies vanished after mcp updateStep Replace the request rate limit and keep everything else | --policies replaces the whole policies array | Always read the current array, merge, then update, as shown in the Azure CLI track. |
Tool rules or policies set in the portal vanished after azd provisionSteps Block the server (Bicep + azd)Turn the server back on (Bicep + azd)Change the limit | A Bicep deployment sends a PUT, which replaces the whole MCP server | Change the azd environment values and run azd provision instead of editing the server in the portal. |
The checkpoint shows "blockList": null, and blocked tools can still be calledSteps Read the outputs and the keyCall a blocked tool | The template uses the preview name blocked, which the GA API drops without an error | Rename the property to blockList and run azd up again |
The deployment fails on requestRateLimitStep Deploy with azd up | The template uses the preview names callsPerPeriod and periodSeconds | Use count and period: "minute" or PT<n>S with 1 to 300 seconds |
A few calls return 502 Bad Gateway right after you change the serverStep Run the burst | The gateway is still rolling out the new configuration | Wait about 30 seconds, then run the script again |
| Policies in the gateway's menu shows "No policies configured"Step Configure the limit | That page lists only gateway-scope policies | Open the MCP server and select its Policies tab |
The call returns "isError": true instead of an HTTP errorStep Call a blocked tool | The gateway denies blocked tools inside the MCP result, with ToolNotAvailable | Expected. Agents read the message and don't retry the tool |
Right after terraform apply or az aigateway mcp update, list-tools still shows the old catalog, or a call fails with AccessBlockedByGatewaySteps Check the catalogCall a blocked tool | The gateway takes 5 to 15 seconds to apply a change to a server's tool controls or access | Wait 15 seconds, then run the script again. |
The query prints "blocked": null although you blocked a toolStep Set the tool controls | From 1.0.0b34, the aigateway extension prints the blocked tools as blockList, the name of API version 2026-11-01 | Query properties.{allowList:allowList,blockList:blockList}, as the step does. |
mcp update fails with ValidationError: "'renewal-period' should be equals to or greater than 1 and equals to or less than 300."Step Replace the request rate limit and keep everything else | period is longer than 300 seconds | Use "minute", or PT<n>S with 1 to 300 seconds. |
az aigateway mcp update has no option for server accessStep Block the server (Azure CLI) | The aigateway extension 1.0.0b34 can't set accessState | Use az rest --method patch, as the step does, or the portal. |
Lab 06: Token limits
Lab 06: Throttle tokens per caller with token limits
| Symptom | Likely cause | Fix |
|---|---|---|
Every request returns 200, never 429Steps Run the burstInspect one throttled response | The limit is on a different model or alias than the one you call, or the window reset between requests | Check that the request model matches an alias of the limited model. Run the burst without pauses. |
429 on the very first requestSteps Run the burstProve the counter is per caller | Another token limit with a lower allowance applies, for example a Gateway scope policy, or the counter is shared because Counter dimensions is empty | Review Policies for every limit that covers the model. Select Identity under Counter dimensions for per-caller counters. |
403 with LlmCostQuotaExceededSteps Run the burstInspect one throttled response | A Cost limit blocked the call, not the token limit | See Enforce cost budgets. |
429 without a gateway token limitStep Run the burst | The backend deployment ran out of TPM | Raise the deployment's capacity in Foundry, or lower per-caller gateway limits so their sum fits. |
| Other policies vanished after the REST callSteps Replace the token limit and keep everything elseRemove the policy (REST API) | PATCH replaces the whole policies array | Always GET, merge, then PATCH, as shown in the REST track. |
ValidationError with target counterKeyStep Replace the token limit and keep everything else | counterKey is a string such as "Identity", uses another value, or is missing | Send an array with identity, ipAddress, or both, for example "counterKey": ["identity"]. |
Lab 07: Cost budgets
Lab 07: Enforce cost budgets with per-caller overrides
| Symptom | Likely cause | Fix |
|---|---|---|
Every request returns 200, never 403Steps Run the requestsInspect one blocked response | The cost limit is on a different model than the alias you call, your key has an override, or the model is so cheap that the calls haven't reached the budget | Check Applies to on the policy details page, use a key without an override, and lower the budget to 0.001 or send more requests. |
403 with LLM cost quota is exceeded on the first requestStep Run the requests | The key already spent more than the new amount in this period (spend is kept when you edit), or a Gateway scope cost limit is exhausted | Review every Cost limit in Policies. Raise the amount, or wait for the next renewal period. |
| An Entra override doesn't applyStep Prove the override and the per-caller counters | An earlier override matched first, the caller used an API key, or you entered a service principal's application ID | Remove the earlier override and add them again in the order you want. Call with an Entra token. Use the service principal's object ID. |
| IP address can't be selected in Counter dimensions, and the panel shows "Remove all overrides before switching to Caller IP address."Step Set the default budget | The policy has overrides | Remove all overrides, or keep Identity. |
429 instead of 403Step Run the requests | A Token rate limit or Request rate limit blocked the call, or the backend ran out of capacity | See Throttle tokens per caller with token limits. |
'set-cost-limit' is misspelled or not recognized by the systemSteps Set the cost limitRemove the cost limit | The aigateway extension is older than 1.0.0b31, the version these labs need for az aigateway model policy | Run the installer again to upgrade it, as described in the Setup guide. |
(ValidationError) Invalid field 'overrides[0].claims' specifiedStep Give specific callers their own budget | The claims shape doesn't match the extension's API version: 1.0.0b34 and later take a list of name and value pairs, and older versions take a map | Run the installer again, then pass claims as a list of name and value pairs, as in the step. |
Every request returns 200 right after you lowered the budgetSteps Lower the budget for the testRun the requests | A policy change takes up to a minute to reach the gateway | Wait a minute and run the script again. The spend so far still counts. |
Invalid field 'overrides[1].claims.tid'Steps Review the templatesDeploy with azd upReview the configurationDeploy with terraform apply | The template sends claims as a map of claim names to values, the shape that API version 2026-11-01 no longer accepts | Send claims as a list of name and value pairs, for example claims: [ { name: 'tid', value: tenantId }, { name: 'oid', value: objectId } ], as in budget-alias.bicep. |
costLimit.counterKey must be an array containing a unique subset of: identity, ipAddress.Steps Review the templatesDeploy with azd up | The costLimit has no counterKey, which API version 2026-11-01 requires | Add counterKey: [ 'identity' ], or [] for one budget shared by every caller. |
Lab 08: Content safety
Lab 08: Screen prompts and responses with content safety
| Symptom | Likely cause | Fix |
|---|---|---|
| Content safety isn't in the Type list of Add policyStep Create the policy | The wizard doesn't offer content safety for AI Gateway resources | Create the policy with the step's az aigateway model update --policies commands, which read, merge, and write the model's policies, then edit its thresholds in Policies. |
'set-content-safety' is misspelled or not recognized by the system, or unrecognized arguments: --remove-content-safetySteps Create the policySet the policyRelax one categoryRemove the policy (Azure CLI) | The aigateway extension 1.0.0b34 removed its content safety options until they match API version 2026-11-01, and an older copy of the step uses them | Use the step's current commands: read the model's policies, merge, and write them back with az aigateway model update --policies. |
Ordinary prompts return 400Steps Run the test promptsRelax one category | A threshold is too strict for your traffic, or another content safety policy, such as a Gateway scope policy, also applies | Review Policies for every content safety policy that covers the model. Move categories from Strict to Balanced and retest with representative prompts. |
| Nothing is blocked, even at Strict, or only Foundry results show upStep Run the test prompts | At the time of writing, AI Gateway resources save the content safety policy but don't block calls with it. Otherwise, the policy is on a different model or alias than the one you call, or a category is Off | Check the policy's scope, and check that the request model matches an alias of that model. Until the gateway enforces the policy, rely on the deployment's content filter, as in Check the Foundry content filter. |
400 with "The response was filtered due to the prompt triggering Azure OpenAI's content management policy" and the code content_filter, or finish_reason is content_filterSteps Run the test promptsRelax one categoryCheck the Foundry content filter | The Foundry deployment's content filter acted, not the gateway | Review the deployment's content filter in Foundry. ModelsBackend models › your model shows it under Content filter. The gateway policy is managed separately. |
400 that isn't about contentStep Run the test prompts | The request itself is invalid, for example an unknown model alias | Read the error message, and check the request body and the alias. |
ValidationError such as contentSafety.violenceSeverity must be one of: Low, Medium, High, None.Step Replace the content safety policy and keep everything else | A severity uses a portal label such as Strict or Balanced | Use the API values from the table in How it works: Low, Medium, High, or None. |
Other policies vanished after the PATCHSteps Replace the content safety policy and keep everything elseRemove the policy (REST API) | PATCH replaces the whole policies array | Always GET, merge, then PATCH, as shown in the CLI and REST tracks. |
Lab 09: Gateway-wide baselines
Lab 09: Set gateway-wide policy baselines
| Symptom | Likely cause | Fix |
|---|---|---|
Every call returns 403Steps Call the model from your IP addressLayer a stricter model policy | Your public IP address changed (new network, VPN, or hotspot), or your traffic to Azure leaves from a different address than the one api.ipify.org reported. A VPN, a corporate proxy, or a client such as Global Secure Access can do that | Check curl -s https://api.ipify.org again and update the IP filter. If you use a proxy, a VPN, or Global Secure Access, ask your network team for the egress ranges, or run the lab from Azure Cloud Shell. Management calls still work, so you can always fix the list. |
check-call still returns 200 after you set 203.0.113.10/32, or still 403 after you restore your rangeStep See the IP filter block a call (optional) | Policy changes take up to a minute to reach the gateway | Wait a minute and run check-call again. |
Teammates, pipelines, or hosted agents get 403Step Watch outcomes in Monitoring | Their egress addresses aren't on the allow list | Add their CIDR ranges, or remove the IP filter until you know them. |
Ordinary prompts return 400Steps Call the model from your IP addressLayer a stricter model policy | Content safety blocked them, from the baseline or from a stricter asset policy | Review every Content safety policy that covers the asset in Policies, and retest with representative prompts. |
429 much earlier than the baseline allowsStep Layer a stricter model policy | A tighter token or request rate limit on the model, alias, or MCP server applies | Review Policies for every limit that covers the asset. The lowest applicable limit wins. |
ipFilter.cidrRanges entries must be valid CIDR rangesSteps Set the baselineSee the IP filter block a call (optional) | An entry has no prefix length, such as 198.51.100.7 instead of 198.51.100.7/32, or --allowed-ips contains a space | Use CIDR notation for every entry, and pass the list without spaces, for example --allowed-ips "[198.51.100.7/32,203.0.113.0/24]". |
'policy' is misspelled or not recognized by the systemSteps Set the baselineRemove the baseline | The aigateway extension is older than 1.0.0b31, the version these labs need for the Gateway scope policy commands | Run the installer again to upgrade it, as described in the Setup guide. |
The PATCH returns 400 about counterKey, scope, count, or periodStep Apply it | counterKey is a string such as "Identity" instead of an array, content safety or the IP filter has "scope": "workspace", or the file uses the preview fields callsPerPeriod and periodSeconds | Use the shapes in infra/baseline-policies.json: "counterKey": ["identity"], count with period, and "scope": "resource" for content safety and the IP filter. |
Policy scope 'resource' is not supported for policy 'tokenLimit' on Workspace. The only supported value is 'workspace'.Step Apply it | A token, cost, or request rate limit at Gateway scope uses "scope": "resource" | Use "scope": "workspace" for those limits, as infra/baseline-policies.json does. Content safety and the IP filter keep "scope": "resource". |
400 when the request rate limit period is hour, day, or longer than PT300SSteps Apply itSet the baseline | The request rate limit accepts minute or a duration from PT1S to PT300S | Use minute or PT<n>S. For longer windows, use a token rate limit or a cost limit. |
| Content safety isn't listed in the Add policy wizardStep Add content safety | The wizard doesn't offer it for AI Gateway resources yet | Run the step's az rest commands, which add content safety to the Gateway scope policies and keep the others, then change categories from the policy's More actionsEdit. |
'set-content-safety' is misspelled or not recognized by the system, or unrecognized arguments: --content-safety Medium or --remove-content-safety trueSteps Add content safetySet the baselineRemove the baseline (Azure CLI) | The aigateway extension 1.0.0b34 removed its content safety options until they match API version 2026-11-01, and an older copy of the step uses them | Use the step's current commands: az rest reads the Gateway scope policies, merges content safety, and writes them back with a PATCH. |
unrecognized arguments: --counter-key Identity from policy set-token-limitStep Set the baseline | The token limit's option is --token-counter-key; only set-cost-limit takes --counter-key | Use --token-counter-key Identity, as the step does. |
A harmful prompt isn't blocked by the gateway, or the 400 comes from the Foundry content filterSteps Add content safetyCall the model from your IP address | AI Gateway resources save the content safety policy but don't enforce it yet | Rely on the Foundry deployment's content filter, as in Screen prompts and responses with content safety. |
With a /32 entry, some calls return 200 and others 403Steps Call the model from your IP addressSee the IP filter block a call (optional) | Your network reaches Azure through a pool of egress addresses, so the address that api.ipify.org reports changes between calls | Allow your network's egress range, for example a /24, or run the lab from Azure Cloud Shell. |
Set ALLOWED_CIDRS ... or Use CIDR notation ... from the scriptSteps Preview the changeApply itSee the IP filter block a call (optional) | ALLOWED_CIDRS is empty, or an entry has no prefix length | Set it to comma-separated CIDR ranges, for example "$MY_IP/32". |
| Other Gateway scope policies vanished after you called the management API yourselfSteps Apply itRemove the baseline | PATCH replaces the whole policies array | Use the az aigateway policy commands, which keep the other policies, or always GET, merge, then PATCH, as the script and the content safety steps do. |
jq: command not foundSteps Preview the changeApply it | jq isn't installed | Install jq, or use the PowerShell commands. |
MethodNotAllowed: "The requested resource does not support http method 'PUT'."Step Choose what to declare | A Bicep template, azapi_resource, or azapi_update_resource declares the default workspace, which accepts only PATCH | Use the Policy as code scripts or the Terraform track, which send a PATCH. |
Set allowed_cidrs to comma-separated ranges in CIDR notation, for example "198.51.100.7/32" for a single address.Step Apply with terraform apply | TF_VAR_allowed_cidrs is empty, or an entry has no prefix length | Set it to CIDR ranges, for example 198.51.100.7/32, as in the step. |
terraform plan shows No changes. after someone changed a Gateway scope policy in the portal or with the Azure CLIStep Change the baseline | The configuration sends a PATCH and doesn't read the policies back, so Terraform can't see the change | Run terraform apply -replace=azapi_resource_action.baseline to send the baseline again. |
Lab 10: Runtime API keys
Lab 10: Issue and rotate runtime API keys
| Symptom | Likely cause | Fix |
|---|---|---|
A new key gets 401Steps Run the checksCheck that the rotated key still works | The value has a space or line break, or the client sends it in another header, such as x-api-key | Copy the value again and send it as Authorization: Bearer <value> or as api-key: <value>. |
400 with Specify either an API key or an Authorization header, not both. | The client sends an api-key header and an Authorization header | Send only one of them. With an OpenAI SDK, pass the key as the SDK's API key and remove the extra header. |
401 with Unauthorized. Missing or invalid Entra ID token, or missing Data.All scope. for an API keySteps Suspend a key insteadCheck that the rotated key still works | The gateway returns the same message for every rejected key: suspended, rotated, deleted, or mistyped | Check state with az aigateway api-key list, and compare the last four characters with list-secrets. |
401 right after Rotate key in the portalSteps Rotate the keyCheck that the rotated key still works | Portal rotation replaces the primary value immediately | Give the client the value from API key rotated successfully. Next time, use the CLI secondary-value flow. |
| "The key was created, but its value couldn't be retrieved."Steps Create the first keyCreate the other two keys | You don't have Microsoft.ApiManagement/aigateways/apiKeys/listSecrets/action | Use Rotate key to get a new value, or ask someone with that permission, such as a Contributor, to read it. |
check-key prints not checked with HTTP 404Steps Move the client to the secondary valueRun the checksCheck that the rotated key still works | The gateway doesn't know the alias in AI_GATEWAY_MODEL, so it never checked the key | Set AI_GATEWAY_MODEL to an alias from Models, for example the one from lab 02. |
A key that worked gets 401, and nobody rotated itStep Check that the rotated key still works | The key is suspended | Check state with the list command, and set it back to active if the suspension is over. |
check-key prints key accepted, but the call failedSteps Move the client to the secondary valueRun the checksCheck that the rotated key still works | The key works, but AI_GATEWAY_MODEL isn't an alias you can call, or a policy blocked the request | Set AI_GATEWAY_MODEL to an alias from Models, for example the one from lab 02. |
Forbidden, ForbiddenByConnection: "Public network access is disabled and request is not from a trusted service nor via an approved private link."Steps Save the current valueRegenerate the unused secondary valueRegenerate the primary value | An Azure Policy in your organization turns off public network access on new key vaults, and keeps it off even after az keyvault update --public-network-access Enabled | Ask your Azure administrator for a policy exemption for the lab's resource group, or run the Key Vault commands from a network that reaches the vault through a private endpoint. |
Forbidden from az keyvault secret set or secret showSteps Regenerate the unused secondary valueRegenerate the primary valueLoad the app-dev key | You have no data-plane role on the vault, or the role assignment hasn't applied yet | Assign Key Vault Secrets Officer on the vault and wait a few minutes. |
| Monitoring shows no caller identities | Monitoring isn't configured, or the time range has no traffic | Complete Monitor usage, cost, and traces, send requests with each key, and widen the time range. |
Lab 11: Entra ID and RBAC
Lab 11: Go keyless with Microsoft Entra ID and RBAC
| Symptom | Likely cause | Fix |
|---|---|---|
401 on every Entra callSteps Call the model with the bearer tokenConfirm the keyless call succeedsRestore access | The token is expired, or it was issued for another audience, such as Azure Resource Manager | Request a new token for the scope https://ai.gateway.azure.com/Data.All (https://ai.gateway.azure.com/.default from an SDK credential). The decoded token's aud is the AI Gateway app ID 18550318-e177-4205-a743-6816d6fcdc3f. |
403 right after you assigned the roleSteps Send a promptCall the model with the bearer tokenRun itRestore access | Role assignments take several minutes to apply | Wait and retry with a fresh token. In the playground, refresh the page once the role applies. |
403 for one model or MCP server onlySteps Send a promptCall the model with the bearer tokenRun it | The role is scoped to a different alias or MCP server, or the role doesn't cover it (Models User on an MCP server, Tools User on a model) | Assign a compatible role at the gateway or at that alias or MCP server. |
403 even though the role is correctSteps Send a promptCall the model with the bearer tokenRun it | A gateway policy (for example a cost limit), the request proxy, or the backend rejected the call | Read the error code in the response body before changing permissions. LlmCostQuotaExceeded comes from a cost limit. |
AuthorizationFailed when you create the assignmentSteps Review and assignCreate the role assignmentUse the helper for other roles and scopesDeploy with azd up | You have Contributor, which can't write role assignments | Ask an Owner, User Access Administrator, or Role Based Access Control Administrator to assign the role. |
DefaultAzureCredential failed to retrieve a tokenStep Run it | No sign-in is available to the credential chain | Run az login, or sign in to the Azure extension in your IDE, then rerun the sample. |
| Your name doesn't appear when you search for itStep Select identities | Name search in the directory is off | Turn on Allow directory search, or paste your object ID (MY_OBJECT_ID), which works without it. |
az role assignment list --scope "$GATEWAY_ID" doesn't show a role you assigned, or a starts_with(scope, '$GATEWAY_ID') filter returns nothingSteps Confirm the assignmentRead the outputsRemove the role and try again | The portal and this lab assign roles at the gateway's /workspaces/default scope, which that list doesn't include, and assignments created with Bicep store /resourcegroups/ in lower case | List with --all --assignee "$MY_OBJECT_ID" and the ends_with(scope, '/aigateways/$GATEWAY') || contains(scope, '/aigateways/$GATEWAY/') filter, as the lab does. |
Credential lifetime exceeds the max value allowed as per assigned policyStep Create a test identity | Your tenant limits client secret lifetimes, and az ad sp create-for-rbac asks for one year | Use the lab's commands, which give the secret a one-day --end-date. |
AADSTS7000215: Invalid client secret providedStep Sign in as the identity | A new client secret takes a few seconds to apply | Wait 30 seconds and sign in again. The lab keeps OPS_SECRET until the sign-in succeeds. |
Subscription '<subscription-id>' not found from az restStep Check what the identity can do | The identity signed in before its role assignment applied, so its profile has no subscription | Wait a minute, run az account list --refresh with the identity's profile, and retry. |
az aigateway commands fail with AuthorizationFailed on Microsoft.ApiManagement/aigateways/readStep Check what the identity can do | The aigateway extension reads the gateway before it lists or changes assets, and AI Gateway Contributor and AI Gateway Developer can't read the gateway resource | Use az rest with the management API, as the lab does. show, update, and delete also work with --ids <asset resource ID>, which skips the gateway read. list and create have no --ids. |
AuthorizationFailed on Microsoft.ApiManagement/aigateways/apiKeys/writeStep Check what the identity can do | AI Gateway Contributor doesn't manage runtime API keys | Expected. Give Contributor on the gateway to identities that manage keys. |
ops aigateway … fails with 'aigateway' is misspelled or not recognized by the systemStep Check what the identity can do | The identity's AZURE_CONFIG_DIR has no extensions, and the Azure CLI looks for them in that folder | Point AZURE_EXTENSION_DIR at your extensions folder (~/.azure/cliextensions by default) in the ops helper, or use az rest as the lab does. |
Model provider, model, or API key lists return [] for an AI Gateway DeveloperStep Check what the identity can do | AI Gateway Developer reads only aliases and MCP servers; the other lists are filtered without an error | Expected. Read aliases or MCP servers, or assign AI Gateway Contributor. |
| The identity can't change Policies at Gateway scopeSteps Grant AI Gateway ContributorChoose the management task | AI Gateway Contributor has no Microsoft.ApiManagement/aigateways/workspaces/write permission | Give Contributor on the gateway for Gateway scope policies. |
| The page says it couldn't be loaded after AssignStep Review and assign | A transient portal error after the assignment is created | Select Retry. The assignment is listed on AccessEntra ID. |
Lab 12: Backend identities
Lab 12: Call backends with managed identities
| Symptom | Likely cause | Fix |
|---|---|---|
403 mentioning permissions on the Foundry account right after the switchStep Call the model through the gateway | The role assignment hasn't propagated, or it's at a different scope than the Foundry account | Wait a few minutes. Check az role assignment list --assignee "$IDENTITY_PRINCIPAL_ID" --scope "$FOUNDRY_ID" -o table. |
| Calls fail after the switch and the role is in placeStep Call the model through the gateway | clientId holds the principal (object) ID, or the identity isn't attached to the gateway | Use the identity's Client ID, and check that az aigateway identity show -g "$RG" -n "$GATEWAY" --query userAssignedIdentities lists the identity. |
az aigateway identity assign fails with an authorization errorStep Attach it to the gateway | You can't assign that identity to resources | Ask for Managed Identity Operator on the identity. |
az aigateway identity show reports that no identity is assigned to the gatewayStep Check it with the CLI | The gateway has no managed identity | Enable it with az aigateway identity assign in Check the system-assigned identity, then grant it Foundry User. |
| Calls fail after you detach an identity or disable the system identitySteps Detach the identity from the gatewayRemove the role assignment and delete the identity | An asset still uses that identity | Check View accessBackend access before you remove an identity, and switch those assets first. |
AuthorizationFailed on roleAssignmentsSteps Check it with the CLIGrant Foundry User on the Foundry accountDeploy with azd up | You can't create role assignments on the Foundry account | Ask for Role Based Access Control Administrator on the account, or have an owner grant the role. |
--query properties.foundry.authentication prints nothing, or the provider's client ID is emptySteps Update the provider's authenticationConfirm which identity the provider uses | From 1.0.0b34, the aigateway extension prints the shape of API version 2026-11-01, where authentication sits directly under the provider's properties | Query properties.authentication, as the step does. |
Lab 13: Monitoring
Lab 13: Monitor usage, cost, and traces
| Symptom | Likely cause | Fix |
|---|---|---|
| "Enable all telemetry signals" on the dashboardSteps Overview: is the gateway healthy, and what does it cost?Confirm all signals are flowing | Metrics, logs, or traces is turned off | Select Open Monitoring settings and set all three endpoints, or disable monitoring and configure it again with Application Insights. |
| "This legacy exporter no longer activates OTLP signals" in SettingsSteps Start the setupConfirm all signals are flowing | The exporter has kind applicationInsights, for example from the reference sample's Bicep or a setup before 2026-08-28 | Select SettingsGeneralDisable monitoring, then select Configure again and choose the same Application Insights resource. |
Apply fails on the role assignment, or az role assignment create returns AuthorizationFailedSteps Apply the configurationLet the gateway publishDeploy with azd upDeploy with terraform apply | You can't create role assignments on the destination | Ask an Owner to grant Monitoring Metrics Publisher on the Data Collection Rule, or to run the setup. |
The telemetry exporter already exists; use telemetry-exporter update.Step Create the telemetry exporter | You ran the step again: from version 1.0.0b33, create doesn't overwrite an exporter | Run the same command with update in place of create. |
(Conflict) A telemetry exporter of kind 'OpenTelemetry' already exists in workspace 'default'.Step Create the telemetry exporter | The gateway already has an OpenTelemetry exporter, for example from the portal wizard. A gateway has one exporter of each type, and an exporter's type can't change | Find it with az aigateway telemetry-exporter list -g "$RG" --gateway-name "$GATEWAY" -o table, then update it, or delete it and create yours. |
The postprovision hook fails with "has no OTLP endpoints after 10 minutes" or "The telemetry exporter deployment failed"Step Deploy with azd up | Azure is still creating the OTLP endpoints, or the new role assignment hasn't propagated yet | Wait a few minutes, then run azd hooks run postprovision. Running azd provision again doesn't help: when the templates haven't changed, azd skips the deployment and its hooks. |
The component has no OTLP endpoints yet. Wait a few minutes, then run terraform apply -replace=module.telemetry.azapi_resource_action.otlp.Step Deploy with terraform apply | Azure is still creating the component's OTLP endpoints | Wait a few minutes, then run the command from the message. |
| "Model token and estimated cost data are incomplete until you add a Prometheus access token"Steps Overview: is the gateway healthy, and what does it cost?Models: who uses which model, and how many tokens? | The portal can't read the metrics workspace with your session | Paste a token from az account get-access-token --resource https://prometheus.monitor.azure.com --query accessToken -o tsv into Prometheus access token. |
Failed to resolve table or column expression named 'OTelSpans'Steps Open LogsRun the starter queries | You opened Logs on the Application Insights resource, which doesn't resolve the OpenTelemetry tables | Open Logs on the Log Analytics workspace: <gateway>-logs, or managed-<insights>-ws in the managed resource group for the portal track. |
| Dashboard and queries are emptySteps Overview: is the gateway healthy, and what does it cost?Run the starter queriesMatch the dashboard to your traffic | No traffic in the selected range, or ingestion is still catching up | Widen the time range, wait a few minutes, and run the traffic script again. |
fail-1 isn't on TracesSteps Compare the three failuresQuery failed operations | A request for an unknown alias names no model on the gateway, so Traces doesn't list it | Find it with query 8 in Logs. |
ai_<insights>_<id>_managed stays with an Azure Monitor workspace, and deleting it fails with ResourceGroupDeletionBlockedSteps Delete the destinationDelete the exporter and the destinationDelete everything azd createdDelete everything Terraform created | Azure hasn't finished deleting the Azure Monitor workspace in it. The state can show Deleting, or Succeeded after a failed delete. A deny assignment blocks deleting it yourself | Leave it to Azure. Its Data Collection Rule and endpoint are already gone, so it receives no data. |
Lab 14: OpenTelemetry chargeback
Lab 14: Attribute tokens and cost with OpenTelemetry
| Symptom | Likely cause | Fix |
|---|---|---|
| No Token claim attributes section in SettingsSteps Open the token claim attributesAdd metrics for the client app (Portal) | The exporter is the legacy applicationInsights kind | Disable monitoring and configure it again, as in Monitor usage, cost, and traces. |
The claims don't change after azd provisionStep Change the claims | The templates haven't changed, so azd skipped the deployment and the postprovision hook that deploys the exporter | Run azd hooks run postprovision. |
The postprovision hook fails with "has no OTLP endpoints after 10 minutes" or "The telemetry exporter deployment failed"Step Deploy with azd up | Azure is still creating the OTLP endpoints, or the new role assignment hasn't propagated yet | Wait a few minutes, then run azd hooks run postprovision. |
The component has no OTLP endpoints yet. Wait a few minutes, then run terraform apply -replace=module.telemetry.azapi_resource_action.otlp.Step Deploy with terraform apply | Azure is still creating the component's OTLP endpoints | Wait a few minutes, then run the command from the message. |
The entra rows return 401 or 403Step Call the gateway as two callers | Wrong token audience, or no AI Gateway Models User role yet | Get the token for the scope https://ai.gateway.azure.com/Data.All, check the role assignment, wait a few minutes, and get a new token. |
| Entra callers show as Unattributed on ModelsSteps Find your callers in MonitoringCompare the report with the dashboard | tid or oid isn't exported to Metrics | Select Metrics for both and send new traffic. Earlier requests stay unattributed. |
| The chargeback query returns no rowsStep Run the query | Telemetry hasn't arrived, the Logs signal is off, or your API reports usage under other names | Wait a few minutes, then run OTelLogs | where TimeGenerated > ago(1h) | take 5 and compare the Attributes with the query. |
| Logs shows table and filter pickers instead of a query editorStep Run the query (Portal) | Logs opened in Simple mode | Switch the mode selector from Simple mode to KQL mode, then paste the query. |
Streamed calls show 0 tokens, and RequestsWithoutUsage isn't 0Steps Run the queryPrice it | Streamed Responses calls report their cost but no token usage | Nothing to fix: GatewayCost still includes them. Count tokens from non-streamed calls, or use the gateway's cost. |
| No Export button above the resultsStep Export it | Logs moved export into the Share menu | Select ShareExport to CSV (all columns). |
Every caller returns 0, or no series at allStep Query token usage for the last day | The query uses increase(), which returns 0 for counters with only a few samples in the window | Use max_over_time(…[1d]), as in the lab. |
ai_<insights>_<id>_managed stays after azd down or terraform destroySteps Delete everything azd createdDelete everything Terraform created | Azure is still deleting the Azure Monitor workspace in it | Leave it to Azure. See the same row in Lab 13. |
Tenant or ClientApp is empty for Entra rowsSteps Run the queryPrice it | The claim wasn't exported to Logs when the request ran | Enable Logs for the claim and send new traffic. |
The Prometheus query returns 401 or 403Steps Query token usage for the last dayQuery estimated cost | Your account can't read the Azure Monitor workspace | Ask for a data read role on the Azure Monitor workspace, such as Monitoring Data Reader, then get a new token. |
Lab 15: Agent frameworks
Lab 15: Connect agent frameworks to models and tools
| Symptom | Likely cause | Fix |
|---|---|---|
401 from the model or MCP endpointStep Run each sample | The key is missing, wrong, or revoked, or the header isn't sent | Check AI_GATEWAY_API_KEY in .env and that the key exists under AccessAPI keys. A value set in your shell overrides .env. |
404 from the MCP endpointStep Run each sample | AI_GATEWAY_MCP_SERVER doesn't match an MCP server name | Copy the name from MCP servers, or the endpoint from the server's page. |
A 4xx error that mentions the modelStep Run each sample | AI_GATEWAY_MODEL isn't a callable alias | Copy the alias exactly from the model's Aliases tab. |
| The agent answers without citing any URLStep Run each sample | The MCP server publishes no tools, or tool controls block them | Run list_tools.py from Federate MCP tools behind one endpoint and check the server's tool controls. |
429 responsesStep Run each sample | A token limit or rate limit on the alias, MCP server, or gateway | Expected when a policy applies. Wait for Retry-After, or raise the limit. |
400: Function tools with reasoning_effort are not supported for <model> in /v1/chat/completionsStep Run each sample | An older copy of a sample calls Chat Completions with tools on a reasoning model that takes function tools only on Responses | Pull the latest samples, which call the Responses API. Or send reasoning_effort: "none" with Chat Completions. |
ModuleNotFoundErrorSteps Run each sampleProve the gateway is in the path | The virtual environment isn't active | Activate .venv and run pip install -r requirements.txt again. |
ToolException: Failed to create MCP session: cannot import name 'Client' from 'mcp'Steps Install the packagesRun each sample | agent-framework-core 1.23.0 or later with mcp 1.x. pip doesn't flag it, because Agent Framework declares mcp only in an extra, and langchain-mcp-adapters still needs mcp<2 | Keep the versions in requirements.txt. To use the latest Agent Framework, install it in a separate virtual environment without LangChain, with mcp>=2.2. |
Lab 16: Foundry hosted agent
Lab 16: Deploy a Foundry hosted agent behind AI Gateway
| Symptom | Likely cause | Fix |
|---|---|---|
ERROR: agent failed (server_error): An internal server error occurred.Steps Ask the hosted agentTrigger the limit | A model or tool call failed inside the agent, and Foundry hides the details | Run azd ai agent monitor gateway-agent --tail 300 and look for Error code, then use the rows below. |
403 "Access denied. You do not have permission to access the requested resource. Ask an administrator to grant the required Azure role."Steps Ask a questionAsk the hosted agent | The caller has no AI Gateway role, or the assignment hasn't applied yet. Locally, the caller is you; in Foundry, it's the agent's identity. | Assign AI Gateway User, as in Give the project and yourself access and Give the agent access, then wait about two minutes. |
404 from the model endpointSteps Ask a questionAsk the hosted agent | AI_GATEWAY_MODEL doesn't match an alias | Copy the alias exactly from the model's Aliases tab, run azd env set AI_GATEWAY_MODEL, then azd deploy --all. |
429 "Token limit is exceeded" in the agent's logs, while local runs still workSteps Ask the hosted agentTrigger the limit | A token limit counted per caller identity, such as the one from this lab: the agent's identity spent its tokens, and you have your own counter | Wait for the time in the message, or remove the policy. |
429 "Your requests to … have exceeded token rate limit"Steps Ask a questionAsk the hosted agent | The Foundry deployment behind the alias ran out of tokens per minute, not the gateway's limit. A fetch of a whole Microsoft Learn article can spend most of a 10,000-tokens-per-minute quota in one call. | Wait a minute, or raise the deployment's capacity. Keep the instruction in main.py that answers from the search results. |
| The agent answers that it can't reach Microsoft Learn, and its log shows "Access denied when connecting to the MCP server at …/toolservers/learn/mcp while calling tool … (HTTP 403 Forbidden)"Steps Ask a questionAsk the hosted agent | The agent's MCP client negotiated version 2026-07-28 with the toolbox, which then sends server/discover to the gateway's learn server. That server supports only the versions of its backend, and answers with 403. | Keep HandshakeClient in main.py, which opens the toolbox connection with the initialize handshake. Your roles are fine when a direct call to the MCP server works. |
429 from tool callsSteps Ask a questionAsk the hosted agent | A request rate limit on the learn server, for example from Control which tools agents can call | Wait for the Retry-After time, or remove the policy from the MCP server. |
AZURE_SUBSCRIPTION_ID is required or no default response for prompt 'Select an Azure Subscription to use'Steps Create an azd environmentProvision the Foundry project | azd runs without prompts in a script, a CI job, or a coding agent, so it can't ask for the subscription | Run azd env new with --subscription "$(az account show --query id -o tsv)", as the lab does, or azd env set AZURE_SUBSCRIPTION_ID before azd provision. |
ERROR: deployment failed: failed invoking event handlers for 'preprovision', getting current environment: rpc error: code = Unknown desc = default environment not foundSteps Provision the Foundry projectDeploy the connection, the toolbox, and the agent | infra/.azure/config.json has no default environment, for example after azd env remove. The Foundry extension's hooks read the default environment and ignore -e. | Run azd env select aigw-lab16, then run the command again. |
RuntimeError: Set AI_GATEWAY_URLSteps Start the agentAsk the hosted agent | An azd environment value is missing | Run azd env set for the missing value, then azd ai agent run or azd deploy --all. |
AGENT_GATEWAY_AGENT_INSTANCE_IDENTITY_PRINCIPAL_ID is emptyStep Find the agent's identity | The agent's identity exists only after the first deployment | Run Deploy the connection, the toolbox, and the agent first. |
| Page couldn't be loaded right after AssignSteps Choose access and scopeAssign the role (Portal) | The role list reloads before the new assignment is readable | Select Retry. The assignment was created. |
| The hosted agent still uses an old valueStep Ask the hosted agent | Foundry reads environment variables when the agent version is created | Run azd deploy --all after every azd env set. |
WARNING microsoft.opentelemetry._distro: Failed to set up A365 OpenAI Agents instrumentation. followed by ModuleNotFoundError: No module named 'agents'Step Read the agent's logs | The hosting runtime tries to instrument the OpenAI Agents SDK, which this agent doesn't use | Expected. Ignore it: tracing still starts (Tracing configured successfully). |
[CodeError] [tool.poetry] section not found during deploymentStep Deploy the connection, the toolbox, and the agent | A pyproject.toml without a lock file was added to src/gateway-agent | Keep requirements.txt as the dependency file, or commit a uv.lock next to pyproject.toml. |
WARNING: gateway-agent (predeploy): AZURE_CONTAINER_REGISTRY_ENDPOINT not set; skipping ACR role check.Step Deploy the connection, the toolbox, and the agent | The agent builds from code on Foundry, so there's no container registry | Expected on the first deployment. Ignore it. |
gateway-tools: Deploying (Waiting for the Foundry project to synchronize.), then after five minutes ERROR: get_toolbox: the Foundry project has not synchronized.Step Deploy the connection, the toolbox, and the agent | You created the azd environment again with the same name soon after azd down --purge, so Foundry recreated a project with the same name. Rerunning azd deploy --all doesn't help for at least 30 minutes. | Run azd down --purge --force, create the environment with a new name, such as aigw-lab16b, then provision and deploy again. |
azd ai agent run can't download packages, with a certificate errorStep Start the agent | A proxy on your network inspects TLS traffic | Set UV_SYSTEM_CERTS=true (UV_NATIVE_TLS=true in older uv versions) so uv trusts your system certificates, then run it again. |
Lab 17: Coding-agent plugin
Lab 17: Build on your gateway with coding agents
| Symptom | Likely cause | Fix |
|---|---|---|
Plugin "ai-gateway" not found in marketplace "azure". Available plugins: aigatewayStep Add the plugin to your agent | The plugin is now named aigateway, but the command uses its earlier name ai-gateway | Install aigateway@azure, and run its commands as /aigateway:discover and /aigateway:build. |
/aigateway:discover isn't recognizedStep Check that the commands are available | The plugin isn't installed or the client hasn't reloaded it | Repeat the install for your agent, then restart it. Check with copilot plugin list in GitHub Copilot CLI or gemini extensions list in Gemini CLI. |
The plugin can't list models, or Azure returns 401 or 403Steps Run discover with your resource IDRead the summary | az is signed in to another tenant or subscription, or your account can't read the gateway | Run az account show, then az account set --subscription <subscription-id>. Ask for read access to the gateway. |
Discover stops with tool calls denied, or asks to run python3Step Run discover with your resource ID | The plugin opens the MCP session with a short Python script, and your agent only allows az and curl | Approve the python3 command, or allow it, for example with --allow-tool 'shell(python3)' in copilot -p. |
| The plugin stops at the credential stepStep Answer the plugin's questions | build never reads, creates, or rotates keys, and you didn't name a credential that it can use | Create a key as in Before you begin, and tell the agent to read it from AI_GATEWAY_API_KEY. Or ask for Microsoft Entra ID, with the AI Gateway Models User role. |
404 Resource not found from the gatewayStep Run the app | The URL is missing the /default/ segment | Use <gateway URL>/default/models/openai/v1 or <gateway URL>/default/toolservers/<name>/mcp. |
unknown_modelStep Run the app | The model name isn't one the gateway publishes, the client uses another protocol than the model supports, or a GitHub Copilot SDK older than 1.0.0 drops the key header | Use the alias that discover reported, with an operation that the alias publishes. If your app uses the GitHub Copilot SDK, pin github-copilot-sdk>=1.0.22, as Use your gateway's models and tools in GitHub Copilot does. |
401 from the gatewaySteps Run the appConfirm the key and model outside the generated codeCall the migrated modelList the migrated MCP tools | The key is missing, deleted, or only seconds old, or the client sends it in another header, such as x-api-key | Send the key as Authorization: Bearer <key>, for models and MCP tools alike. A new key can take up to a minute to work, so wait and retry. Create a new key if you deleted it. |
400 "Specify either an API key or an Authorization header, not both"Step Run the app | The client sends the key in api-key and also an Authorization header, as SDKs do when you pass a placeholder key | Send the key only as Authorization: Bearer <key>. For the model endpoint, pass the runtime key as the SDK's API key and drop the extra api-key header. |
KeyError: 'AI_GATEWAY_API_KEY'Step Run the app | .env wasn't loaded, often because PowerShell saved it with a byte order mark | Export the variable in your shell, or save .env as UTF-8 without a byte order mark. |
429Steps Run the appConfirm the key and model outside the generated code | A token limit or the backend's quota | The wiring works. See Throttle tokens per caller with token limits. |
A tool server shows 0 toolsStep Read the summary | The server exposes no tools, or tool availability hides them | Check the server's tools in the portal, or see Federate MCP tools behind one endpoint. |
| The summary lists the tool servers but not their toolsStep Read the summary | Discover never reads a key's secret, and you didn't name a credential, so it can't open an MCP session | Export the key in AI_GATEWAY_API_KEY and add "use the key in the AI_GATEWAY_API_KEY environment variable" to the command. |
| The summary doesn't list the gateway's API keysStep Read the summary | Discover lists key metadata only when you ask | Ask "Which API keys does the gateway have?". It lists their names and states, never their values. |
build reports that it didn't verify the requestStep Answer the plugin's questions | build asks before a verification request that costs tokens, and you declined, or the agent can't ask, as with copilot -p | Approve the call when build asks, or say in the description that you approve one model call. Or run the app yourself, as in Run the app. |
discover-tools prints needsAuthorization mcp:learn-mcp via none and Nothing was called.Step Inventory the AI assets | Listing a server's tools calls the server, maybe with a subscription key, so discover-tools waits for your authorization | Run it again with --allow-secret-use mcp:<server>, as the step does. |
The summary shows a model provider with auth=undetermined, and the plan has no identity.assign step and no Foundry role prerequisiteStep Inventory the AI assets | API Management signs in to Foundry with an authentication-managed-identity policy on the API, which migrate-apim 0.1.0 doesn't read (#62) | Use the Bicep or Terraform track: their templates give the gateway an identity and the Foundry User role. On the direct track, after the gateway phase, run az aigateway identity assign with --system-assigned true as in Check the system-assigned identity, then create the provider as in Publish the model, which grants the role. |
[BLOCK] noTarget: The plan does not say where it deploys to. and Refusing to apply while a blocking finding stands.Step Preflight the migration | The plan has no target | Run plan with --target, as in Point the plan at a new gateway. |
[BLOCK] unsettled: 6 item(s) still need a decisionStep Preflight the migration | plan.json doesn't hold your decisions, or you didn't run plan again after you recorded them | Record the decisions as in Review the migration strategy for each asset, then run plan again. |
[BLOCK] nameCollision: A Microsoft.ApiManagement/service named '<gateway>' exists in the same resource group.Step Preflight the migration | An API Management instance in the resource group has the gateway's name, and commands that find resources by name could reach it | Choose another gateway name, and run plan again with its --target. |
error: phase '<phase>' has not been rehearsed.Steps Create the gateway and its identityPublish the modelFederate the MCP serverCarry over the token limit and the keys | You ran the phase with --execute before its rehearsal | Run the phase without --execute first. The step's && does it for you. |
The providers phase stops with (ValidationError) The displayName property is required.Step Publish the model | migrate-apim 0.1.0 sends no display name, which az aigateway 1.0.0b34 requires (#62) | Create the provider and the model with az aigateway, as the step does, then run the models and aliases phases. |
--next-phase runs providers again after modelsStep Publish the model | migrate-apim 0.1.0 picks the first phase that isn't complete (#62) | Name each phase with --phase, as the migrate-phase shortcut does. |
The toolServers phase stops with the object exists but does not match the desired state, or with This MCP registration already exists; use mcp update. when you run it againStep Federate the MCP server | migrate-apim 0.1.0 compares the server with the shape that an older az aigateway returned (#62) | If az aigateway mcp show prints Succeeded, the server is fine. Go on with the next phase. |
--status doesn't list acknowledged: prereq:foundry-rbacStep Check the migration status | With migrate-apim 0.1.0, --acknowledge without --preflight prints the status but doesn't save the acknowledgement in journal.json | Run apply --preflight --acknowledge prereq:foundry-rbac, as the step does. |
404 "Resource not found", although every phase reads back as verifiedStep Call the migrated model | The Foundry provider's endpoint ends in /openai, as API Management's backend URL does (#62) | Create the provider with --foundry-resource-id instead, as in Publish the model: it uses the account's root endpoint. |
403 from the backendStep Call the migrated model | The gateway's identity lacks Foundry User on the Foundry account, or the assignment hasn't propagated yet | Wait a few minutes. If it persists, check that the role is assigned to the new gateway's principal ID on the Foundry account. |
503 and {"error":"App port is not ready"}Steps Call the migrated modelList the migrated MCP tools | The new gateway's runtime is still starting | Wait a few seconds and run the call again. |
Lab 18: Private networking
Lab 18: Lock down backend traffic with private networking
| Symptom | Likely cause | Fix |
|---|---|---|
| The create dialog flags the subnet, or the create request fails with a subnet errorSteps Choose private routingCreate the gateway in the integration subnetDeploy with azd up | The subnet is in another subscription or region, smaller than /27, not delegated to Microsoft.App/environments, or used by another gateway or resource | Fix each item. The subnet's NSG needs no custom outbound rules. |
The create request fails with AIGatewaySubscriptionGatewayLimitExceededSteps Create the gateway in the integration subnetDeploy with azd up | A subscription holds at most five gateways per region | Delete a gateway that you no longer need in that region, or create the lab's gateway in another region. Deleted gateways aren't soft-deleted, so the slot frees up once the deletion finishes. |
The gateway's provisioning fails with ActivationFailed and TooManyRequestsSteps Create the gateway in the integration subnetDeploy with azd up | The service couldn't finish activating the gateway | Delete the failed gateway, and create it again with a new name. |
The create request fails with AIGatewaySubnetNotDelegated: "Virtual network integration with subnet ID '…' requires the subnet to be delegated to 'Microsoft.App/environments'."Steps Choose private routingCreate the gateway in the integration subnetDeploy with azd up | The subnet isn't delegated, or it's delegated to another service, such as Microsoft.Web/serverFarms | Delegate the subnet to Microsoft.App/environments, then create the gateway again. |
| "Virtual network integration for Azure AI Gateway can only be configured during creation."Steps Create the gateway in the integration subnetDisable public network access on Foundry (Bicep + azd)Remove the integration subnetAdd the integration subnet back | An older service version, or an older Azure CLI extension, that couldn't change private routing on an existing gateway | Upgrade the extension by running the installer again, as described in the Setup guide, then use NetworkingOutbound in the portal or az aigateway network update --subnet (or --disable-vnet-integration). The change takes a minute or two. |
| Model calls through the new gateway fail after you turn off Foundry public accessStep Call the model through the gateway | The gateway resolves Foundry to its public IP: the zones aren't linked to the integrated virtual network, the private endpoint has no DNS zone group, or the gateway was created without private routing | Check the A records and zone links, and confirm that properties.backend.subnet.id points to the integration subnet. |
The gateway from the earlier labs, or another app, now gets 403 from FoundryStep Disable public network access on Foundry | It reaches Foundry over the internet, and Foundry's public access is disabled | Expected. Use a dedicated Foundry resource for this lab, or re-enable public access when you finish. |
The first call after the import returns 401 or 403 from the backendSteps Create a runtime API keyCheck the model and the keyCall the model through the gateway | The Foundry User role assignment hasn't propagated yet | Wait a few minutes, and confirm that the role is assigned to the new gateway's principal ID on the Foundry resource. |
Model calls return 403 "Public access is disabled. Please configure private endpoint."Steps Turn off virtual network integrationRemove the integration subnet | Without private routing, the gateway calls Foundry over the internet, and Foundry's public access is off | Expected. Add the integration subnet back. |
The update fails with AIGatewaySubnetNotDelegated, or the portal shows "Not delegated to Microsoft.App/environments"Steps Add the integration subnet backTurn virtual network integration back on | The subnet isn't delegated to Microsoft.App/environments | Delegate the subnet, or pick snet-aigw-integration. |
azd provision finishes without deploying, and the gateway still has no subnetStep Provision the declared subnet again | azd skips a deployment when the template and parameters haven't changed, so it doesn't see changes made in the portal or the CLI | Run azd provision --no-state. |
Every call to the gateway returns 403 {"error":"Public network access is disabled for this resource."}Step Disable public network access | Public network access is Disabled, and the call doesn't come through an inbound private endpoint | Expected. Turn it back on (NetworkingInbound, az aigateway network update --public-network-access Enabled, or PATCH {"properties":{"publicNetworkAccess":"Enabled"}}), or call from a client that reaches the gateway through a private endpoint. |
Creating an inbound private endpoint with the group ID Gateway fails with SubscriptionNotRegisteredForFeature for Microsoft.Network/AllowPrivateEndpoints, or az feature show reports it as PendingSteps Create the gateway's private endpointTurn on the gateway's private endpoint | Private endpoints for AI Gateway are in preview, and Microsoft hasn't enabled them for your subscription. Registering the feature yourself leaves it Pending | Create the Azure support request in Request access to private endpoints, with the gateway's name and resource ID, and wait until az feature show reports Registered. Until then, skip to Turn off the gateway's public network access. |
| Create a private endpoint opens the Azure portal in another directory, or the wizard proposes another region or virtual networkStep Create the gateway's private endpoint | The Azure portal opens in your default directory, and the wizard defaults to its own region and the first virtual network in that region | Switch to the gateway's directory and select Create a private endpoint again. On Basics, select the gateway's region, and on Virtual Network, select vnet-aigw and snet-private-endpoints. |
The private DNS zone has no A record for the gateway, and the test client doesn't resolve the gateway to a 10.20.1.x IPStep Link the gateway's private DNS zone | The zone is privatelink.azure-api.net, as in the product docs. The gateway's record goes to the zone that follows its host name | Read the zone from the gateway's private link resource (requiredZoneNames), for example privatelink.eastus2-01.ai.gateway.azure.com, create it, link it to vnet-aigw, and add it to the endpoint's DNS zone group. |
The deployment of private-client.bicep fails with MissingSubscriptionRegistration for Microsoft.ContainerInstanceStep Call the gateway from inside the virtual network | Azure Container Instances isn't registered in the subscription | Run az provider register --namespace Microsoft.ContainerInstance --wait, then deploy again. |
az container logs prints nothing, or only the DNS: lineSteps Call the gateway from inside the virtual networkDisable public network access | The test client hasn't finished its call yet | Wait a minute and run az container logs again. To call again, run az container start -g "$RG" -n aci-private-client -o none, which waits for the run. |
The direct call returns 401 PermissionDenied instead of 403Step Call Foundry directly from your workstation | You have no data-plane role on Foundry, and Foundry checks it before the network rule | Expected: the call is blocked either way. To see the 403, assign Cognitive Services OpenAI User and wait; it can take 30 minutes or more, and calls can alternate between 401 and 403 meanwhile. |
| Deleting the resource group or the virtual network fails because the subnet is in useSteps Delete the lab's resourcesDelete everything azd created | The service is still releasing the gateway's integration subnet | Wait a few minutes after the gateway is gone, then delete again. |
terraform plan still says No changes. after you removed private routing, and terraform apply doesn't add it backSteps Remove the integration subnetAdd the declared subnet back | The gateway no longer returns backend.subnet, and azapi skips properties that the service doesn't return, so it doesn't see the change | Add the subnet back with az aigateway network update, as in the step, with --subnet "$(terraform output -raw ai_gateway_subnet_id)". |
terraform destroy can't delete snet-clients or the virtual network, because the subnet is in useStep Delete everything Terraform created | The test client aci-private-client isn't in Terraform's state and still uses snet-clients | Run az container delete -g "$RG" -n aci-private-client --yes, then run terraform destroy again. |
az network private-link-resource list --id "$GATEWAY_ID" fails with "Resource type must be one of …"Step Link the gateway's private DNS zone | The core command doesn't support Microsoft.ApiManagement/aigateways, and the aigateway extension 1.0.0b34 has no private link resource command | Read the zone with az rest, as the step does. |
Lab 19: Tool threat protection
Lab 19: Protect tool calls with Agent 365 threat protection
| Symptom | Likely cause | Fix |
|---|---|---|
Every protected call returns ToolEvaluationIdentityRequiredStep Call the protected tool with your user token | The call has no bearer token, or the token isn't a delegated user token. App-only tokens and runtime keys can't be used for OBO. | Sign in with az login as a user, get a token for the scope https://ai.gateway.azure.com/Data.All, and send it as Authorization: Bearer. Check that the token has an scp claim. |
initialize or tools/call returns HTTP 401 or 403 ("Access denied") with a bearer tokenSteps Call a tool with your user tokenCall the protected tool with your user token | The user has no data-plane role on the MCP server, or the assignment hasn't applied yet | Assign AI Gateway Tools User at the MCP server or gateway scope and wait a few minutes |
The portal reports "1 role assignment failed." without a reason, or az role assignment create or delete fails with AuthorizationFailedSteps Grant yourself the AI Gateway Tools User roleRemove the role assignment | Your account can't manage role assignments on the gateway: Contributor, or a role whose permission to assign roles carries a condition that excludes this role | Ask someone with Owner, User Access Administrator, or Role Based Access Control Administrator on the gateway or above to assign or remove the role. An assignment you can't remove goes away when you delete the gateway. |
Calls return ToolEvaluationUnavailable ("could not be evaluated by the security service")Step Call the protected tool with your user token | The first evaluation right after you add the policy, or the token exchange or evaluation failed, timed out, returned an invalid verdict, or the evaluator was unavailable | Wait a minute and call again. If it persists, confirm the Microsoft Defender or Microsoft Purview license and the Defender policy mode with your security team, and share the correlationId. |
A benign call returns ToolEvaluationDeniedStep Call the protected tool with your user token | Defender or Purview flagged the request or the result | Review the call with your security team using the correlationId. If the tool already ran (after-invocation stage), don't retry it automatically. |
| You can't add a second tool threat protection policy to the serverSteps Select the MCP serverChoose the evaluation stages | Each MCP server can have only one tool threat protection policy | Keep the existing policy, or delete it and add a new one with the stages you want |
| Tool calls got noticeably slowerStep Call the protected tool with your user token | Each enabled stage adds an evaluation round trip | Keep both stages where you need them, and use only Before tool invocation where results are trusted |
A script stops with HTTP 503 and {"error":"App port is not ready"}Steps Call a tool with your user tokenCall the protected tool with your user token | The gateway is still starting after azd up or azd provision | Wait 30 seconds and run the script again. |
A policy you added in the portal vanished after azd provisionSteps Turn on both stagesConfirm the policy | A Bicep deployment sends a PUT, which replaces the whole MCP server | Change the azd environment values and run azd provision instead of editing the server in the portal. |
Lab 20: Agent 365 governance
Lab 20: Govern MCP servers with Agent 365
| Symptom | Likely cause | Fix |
|---|---|---|
| The gateway's MCP servers don't appear under AgentsToolsStep Find your MCP server | Consent isn't granted, or the instance is on an API Management v1 tier | Have a tenant administrator check the consent box and turn on Azure API Management service and Azure AI Gateway under AgentsSettingsGateways. Use a v2 tier or the AI Gateway tier. |
| AgentsSettings has no Gateways item, or AgentsTools is emptyStep Grant consent | The tenant has no Agent 365 license, or you aren't a tenant administrator | Ask a tenant administrator with Agent 365 to grant consent. You can still practice the gateway-side controls: Blocked tools and MCP server access. |
| The gateway's server doesn't appear yet right after you grant consentStep Find your MCP server | Agent 365 syncs the inventory on a schedule | Wait about 10 minutes and refresh AgentsTools. |
| Two or more Microsoft Learn rows show upStep Find your MCP server | The Catalog rows come from the Agent 365 catalog, not from your gateway | Select the row whose Source is AI Gateway and whose Publisher is your gateway's name. |
| About this MCP Server reads "No description provided." and the tools have no descriptionsStep Find your MCP server | Agent 365 doesn't sync the descriptions from the gateway | Expected. The names, the server URL, and the status are synced. |
| The server's Activity tab shows no dataStep Find your MCP server | The calls in this lab use a runtime key, so they aren't attributed to an agent | Expected. The tab shows calls that agents make. |
| The tool still runs, or stays blocked, after you flip its switchSteps Turn off the tool in Agent 365Turn the tool back on | The switch doesn't apply until you save | Select Save and wait for Changes saved., which takes up to about a minute. |
| The server's Tools tab can't list its toolsStep Turn off the tool in Agent 365 | The server doesn't have a static list of allowed tools: it uses Allow all tools or a wildcard pattern | In the AI Gateway portal, choose Match tools by name pattern in the Tools section of the server's Edit page and enter exact tool names without wildcards. Otherwise, block tools in the AI Gateway portal. |
| You can't block individual toolsStep Turn off the tool in Agent 365 | The instance is on an API Management v2 tier | Tool-level blocking needs Azure AI Gateway. Block the whole server instead. |
| A blocked tool still runs for some agentsStep Call the blocked tool | Those agents call the MCP server directly, not through the gateway | Point every agent at the gateway's MCP endpoint and restrict direct access to the backend |
The blocked tool is still in tools/listStep Call the blocked tool | Blocking a tool doesn't hide it; only calls to it are denied | Expected. Check result.isError and the ToolNotAvailable denial code on tools/call. |
| An agent keeps retrying a blocked toolStep Call the blocked tool | The agent ignores isError | Check result.isError and the ToolNotAvailable denial code, and stop retrying that tool |
initialize returns HTTP 200 with a JSON-RPC error -32000 "Access to MCP server 'data.code AccessBlockedByGatewayStep Try to connect | The server is blocked in Agent 365, or its MCP server access is Blocked in the AI Gateway portal. Both set the same access state | That's the expected block on an AI gateway. The 403 in the Agent 365 article applies to API Management tiers. To unblock, unblock the server in Agent 365, or set MCP server access to Active in the AI Gateway portal. |
Calls still return AccessBlockedByGateway right after you unblockStep Unblock the server | The access state takes a few seconds to reach the gateway | Wait about 30 seconds and run the script again. |
| Calls still fail after you unblockSteps Unblock the serverConfirm the server works end to end | Another control blocks them, such as the gateway's Blocked tools, MCP server access, or a policy | Check the Basic info and Tools sections of the server's Edit page, and its policies, in the AI Gateway portal |
Lab 21: Enterprise reference
Lab 21: Deploy an enterprise reference architecture
| Symptom | Likely cause | Fix |
|---|---|---|
The deployment fails with InsufficientQuota, or the model or version isn't availableSteps DeployDeploy with terraform applyRun it | The region has no GlobalStandard quota for gpt-5-mini, or doesn't offer that version | Lower modelCapacity, request quota, or change modelName, modelVersion, or --location in your parameter file and command. |
The gateway's provisioningState ends in FailedSteps DeployDeploy with terraform applyRun it | Azure hit an activation or managed identity conflict, often after a gateway with the same name was deleted moments before | Delete the failed gateway, wait about three minutes for the identity to be cleaned up, and deploy again. |
The smoke test prints Waiting for the model route until it fails with 401, 403, or 404Steps Run itRun the smoke test | The new Foundry User role assignment or the model route hasn't propagated yet | Wait a few minutes and run the smoke test again. If it still fails, run the backend identity checks in Verify. |
| The telemetry script stops with "has no OTLP endpoints after 10 minutes"Steps Connect telemetryRun it | Azure hasn't finished creating the component's OTLP endpoints | Wait a few minutes and run the script again. If it still fails, check that the component's AzureMonitorWorkspaceIngestionMode is Enabled. |
terraform apply fails on module.telemetry.azapi_resource_action.otlp with Application Insights has no OTLP endpoints or Data Collection Rule yet.Step Deploy with terraform apply | Azure hasn't finished creating the component's OTLP endpoints | Wait a few minutes, then run terraform apply -var-file dev.tfvars -replace=module.telemetry.azapi_resource_action.otlp. |
The telemetry script fails with AuthorizationFailed on roleAssignmentsSteps Connect telemetryRun it | The identity that runs it can't create role assignments on the Data Collection Rule | Use the same rights as the deployment: Owner, or Role Based Access Control Administrator, on the subscription. |
aigw-telemetry fails with "Telemetry exporter kind cannot be changed after creation."Steps Connect telemetryRun it | An exporter with the same name has another type (telemetryExporterType), and the script couldn't read it to replace it, for example because the aigateway extension isn't installed | Install the aigateway extension, then run the script again. It replaces the exporter. |
apply-baselines fails with a validation error from the PATCH, such as Request periods must be minute, hour, day, or a positive whole-second time-only ISO 8601 duration.Steps Apply the gateway baselineRun it | A field in baselines.json doesn't match the 2026-11-01 contract, for example a requestRateLimit with callsPerPeriod and periodSeconds instead of count and period, or counterKey as a string instead of an array such as ["identity"] | Create the policy once in the portal at Gateway scope, read the workspace with az rest and API version 2026-11-01, and copy that shape into baselines.json. |
Every runtime call returns 403 with Public network access is disabled for this resource.Step Run the smoke test | The gateway's public network access is Disabled. A deployment that doesn't set it keeps the current value, so redeploying doesn't turn it back on | Call from a network that reaches the gateway's private endpoint, or PATCH the gateway with {"properties":{"publicNetworkAccess":"Enabled"}} and API version 2026-11-01. |
azure/login fails with AADSTS700213: No matching federated identity record found for presented assertion subject 'repo:<github-org>@<org-id>/<github-repo>@<repo-id>:environment:dev'Steps Trust your GitHub environmentsRun it | The token subject doesn't match a federated credential. Repositories created after July 15, 2026 use immutable subject claims, which add the owner and repository IDs, so a credential with the subject repo:<github-org>/<github-repo>:environment:dev doesn't match | Delete the credentials and add them again with the subject prefix from gh api repos/<github-org>/<github-repo>/actions/oidc/customization/sub --jq .sub_claim_prefix, as in Trust your GitHub environments. Also check that the GitHub environment name matches the job's environment:. |
Saving Required reviewers on prod fails, or gh api returns HTTP 422 with Please ensure the billing plan supports the required reviewers protection ruleStep Create the GitHub environments | On the GitHub Free, Pro, and Team plans, required reviewers are only available for public repositories | Use a public repository or a GitHub Enterprise plan, or create prod without required reviewers for the lab. |
Lab 22: Priority and weighted routing
Lab 22: Route an alias across models by priority and weight
| Symptom | Likely cause | Fix |
|---|---|---|
"Select models with at least one common API format." in the portal, or Active-priority models must share at least one supported endpoint. from the CLI, REST, or BicepSteps Create the chat aliasUpdate the prioritiesProvision the priority strategy | The backend models at the active priority don't share an endpoint, for example an Anthropic Messages model and an OpenAI Chat Completions model | Put models that share an endpoint in the same alias. supportedEndpoints on each model lists its endpoints. |
"Weight must be a whole number from 1 to 100.", Each target's weight must be a whole number from 1 to 100., or weight must be between 1 and 100.Steps Create the chat aliasEdit the aliasUpdate the priorities | A weight is 0, negative, or above 100 | To stop traffic to a model, raise its priority number or remove it. |
"Priority must be a whole number from 0 to 100.", Each target's priority must be a whole number from 0 to 100., or priority must be between 0 and 100.Steps Create the chat aliasEdit the aliasUpdate the prioritiesPromote the new model | A priority is outside 0 to 100 | Use 0 for the preferred tier and higher numbers for the others. |
"Each backend model can appear only once." or backendModels must not contain duplicate model identities.Steps Create the chat aliasUpdate the prioritiesPromote the new model | The same backend model is listed twice | Combine the entries and adjust the weight. |
| All requests go to one modelSteps Create the chat aliasRead the outputs and the key | The models have different priorities, so only the lowest number receives traffic | Give the models the same priority to split by weight. |
Requests from a client that sends x-ai-session-id, x-claude-code-session-id, or session-id keep landing on one model, so the split isn't 80/20Steps Create the chat aliasRead the outputs and the key | Enable session affinity is on by default in the Create alias wizard | Clear Enable session affinity on the Session affinity step, or edit the alias. The alias's Overview then shows Session affinity Disabled. |
404 with DeploymentNotFoundSteps Create the chat aliasRead the outputs and the keyPromote the new model | The candidate backend model points to a Foundry deployment that doesn't exist | Create the deployment, or fix deployment.resourceId on the backend model. |
The first requests of the count return HTTP 404 model_not_foundSteps Create the chat aliasRead the outputs and the key | The new alias takes a few seconds to reach the gateway | Wait a few seconds and run the count again. |
Some requests show HTTP 429 rate_limit_exceededSteps Read the outputs and the keyCount the requests again | The lab's gpt-5-mini deployment allows 10 requests per minute, and the counts and earlier steps use them up | Wait a minute and run the count again. |
One of the first requests shows gpt-6.1-sol-…Step Count the requests again | The priority change takes a few seconds to reach the gateway | Run the count again. Every request then shows gpt-5-mini-…. |
429 from the preferred modelSteps Count the requests againPromote the new model | The preferred deployment is out of capacity. Without failure rules, the gateway returns the error and doesn't try the next priority | Add a failure rule for 429 to the alias, as in Fail over to a backup model when a backend fails, or retry in the client, add capacity, or spread traffic with weights at the same priority. |
Lab 23: Streaming responses
Lab 23: Stream responses and count every token
| Symptom | Likely cause | Fix |
|---|---|---|
The first request after azd up or terraform apply returns {"error":"App port is not ready"}Steps Watch the raw eventsStream with the script | The new gateway's runtime is still starting | Wait a few seconds and send the request again. |
IndexError: list index out of range in Python, or Cannot read properties of undefined in TypeScriptSteps Stream from codeRun the sample | The code reads choices[0] on the prompt filter chunk or the usage chunk, which have no choices | Skip chunks whose choices array is empty, as the samples do. |
| The whole reply appears at onceSteps Watch the raw eventsStream with the script | curl runs without -N, PowerShell uses Invoke-RestMethod, or a proxy buffers the response | Use curl -N, read the stream with HttpClient as stream-chat.ps1 does, and turn off response buffering in your proxy. |
| No usage chunk at the end of the streamSteps Ask for usage at the end of the streamRun the sampleStream a burst with usage | The client didn't set stream_options.include_usage | Set it when your app needs the numbers. The gateway counts the tokens for its limits either way. |
400 with The 'stream_options' parameter is only allowed when 'stream' is enabled.Step Ask for usage at the end of the stream | The request sets stream_options without "stream": true | Send both, or neither. |
429 JSON response instead of a streamSteps Stream a burst without asking for usageStream a burst with usageInspect a throttled streaming request | A token limit on the alias, the model, or the gateway is exhausted | Wait for Retry-After seconds. The limit is checked before the stream starts, so no partial reply is lost. |
The SDK sample never shows a 429Step Run the sample | The OpenAI SDKs retry 429 responses automatically and honor Retry-After | That's the behavior you want in an app. To see each 429, create the client with max_retries=0 (Python) or maxRetries: 0 (TypeScript). |
Every request in the burst returns 200Steps Stream a burst without asking for usageStream a burst with usage | The policy isn't active yet, or the burst used another alias | Check that STREAM_ALIAS matches the alias with the limit, wait a minute, and run the burst again. |
Lab 24: GitHub Copilot BYOK
Lab 24: Use your gateway's models and tools in GitHub Copilot
| Symptom | Likely cause | Fix |
|---|---|---|
| The import adds deployments from other Foundry accountsStep Import the deployment | The Models grid preselects the deployments of every account in the selected subscriptions | In Resource groups, select only your Foundry account's group, then select only gpt-6.1-sol. |
| "Backend isn't compatible with all configured MCP versions. The backend reports support for 2025-06-18, but this MCP server requires 2025-11-25…"Step Register the Learn server | More than one version is selected in Supported MCP versions, and Microsoft Learn supports only 2025-06-18 | Select only 2025-06-18. |
A learn server exists even though you left the wizardStep Register the Learn server | Next on the Backends step saves the server | Finish the wizard, or delete the server in MCP servers. |
mcp create fails with Operation returned an invalid status 'Bad Gateway'Step Register the Learn server | Another MCP server on the gateway already uses the display name Microsoft Learn, in any letter case, for example lab 04's server under another name | Reuse that server and its endpoint, or pass another --display-name. |
InsufficientQuota: The deployment requires 100 quota units, but only 0 quota units are availableSteps Deploy the model in FoundryDeploy with azd up | The subscription's Global Standard capacity for gpt-6.1-sol in the region is used up | In the Azure CLI track, deploy with --sku-name DataZoneStandard. With azd, run azd env set BYOK_MODEL_SKU DataZoneStandard and azd up again. Or deploy in another region. |
HTTP 401 or "Authentication failed with provider"Steps Pick the model and run an agent taskAsk a quick questionRun a sampleStart a session on gpt-6.1-solTurn Copilot back on | The key is wrong, suspended, or deleted, or the new key isn't active yet | Check the key's state in AccessAPI keys, wait a minute after you create it, and paste it again. |
400 on every request from a custom clientStep Run a sample | The request sends both Authorization and api-key | Drop the api-key header. Copilot sends Authorization: Bearer only. |
404, or a "model not found" errorSteps Pick the model and run an agent taskAsk a quick questionRun a sampleStart a session on gpt-6.1-sol | The model name isn't an alias on the gateway, for example the portal import's <foundry-account-name>/gpt-6.1-sol | Publish the gpt-6.1-sol alias, or set the model ID to the alias that exists. |
| The model doesn't appear in VS Code or the appSteps Pick the model and run an agent taskStart a session on gpt-6.1-sol | The gateway has no /models list to discover, or VS Code hasn't reloaded the file | Add gpt-6.1-sol by hand, then run Developer: Reload Window in VS Code. |
| The model is missing from the picker in Agent modeStep Pick the model and run an agent task | toolCalling is missing or false | Set "toolCalling": true on the model. |
| VS Code or the app doesn't offer your own models at allSteps Add the gateway as a custom endpointAdd the gateway as a model provider | Your Copilot Business or Enterprise organization hasn't allowed custom models | Ask an administrator to enable them. |
| Replies are empty or cut off, or tool calls failSteps Pick the model and run an agent taskAsk a quick questionRun a sampleStart a session on gpt-6.1-sol | The surface uses Chat Completions with a reasoning model | Set the API to Responses: apiType in VS Code, COPILOT_PROVIDER_WIRE_API=responses for the CLI, wireApi in the SDK. |
429 Too Many RequestsSteps Pick the model and run an agent taskRun an agent task | Copilot used the key's 100,000 tokens for this minute | Wait for the Retry-After interval, or raise the token limit. |
| The CLI's first interactive prompt doesn't respondStep Run an agent task | A slow first connection | Press Ctrl+C and send the prompt again. |
| Two GPT-6.1 Sol entries in VS Code's model pickerStep Pick the model and run an agent task | Copilot's built-in model and yours share a name | Give yours a distinct name, such as GPT-6.1 Sol (AI Gateway). |
The MCP server doesn't start, or returns 401Steps Start the serverCheck the tools in a sessionRun a sample with the toolsCheck the server | The key is sent in another header, or ${AI_GATEWAY_API_KEY} is empty where the CLI or the app runs | Send the key as Authorization: Bearer ${AI_GATEWAY_API_KEY}. Export AI_GATEWAY_API_KEY in the terminal, or make it available to the app and reopen it. |
jq: error … Cannot iterate over null right after you create the serverStep Test the MCP endpoint | The new learn server is still taking effect, so the reply has no result | Wait a minute and run the step again. |
copilot -p doesn't use the Learn tools, or says they were deniedStep Use the tools in one-shot mode | One-shot mode can't ask for approval | Add --allow-tool='ai-gateway-learn'. |
A task with the Learn tools hits 429 or uses many tokensSteps Run a task with the toolsUse the tools in one-shot modeRun a sample with the tools | Each Learn result is 25 to 50 KB | Ask for fewer lookups, publish fewer tools, or raise the token limit. |
| VS Code lists MCP servers that you didn't addStep Check the tools | VS Code discovers servers configured in other apps | Set chat.mcp.discovery.enabled to false. |
Lab 25: Claude Code
Lab 25: Use your gateway's models and tools in Claude Code
| Symptom | Likely cause | Fix |
|---|---|---|
'import' is misspelled or not recognized by the systemStep Import the Claude deployments | The aigateway extension is older than 1.0.0b30, the first version with model-provider import | Run the installer again, then check the version with az aigateway version. |
| The import adds deployments from other Foundry accountsStep Import the Claude deployments | The Models grid preselects the deployments of every account in the selected subscriptions | In Resource groups, select only your Foundry account's group, then select only the Claude deployments. |
404 with {"error":{"code":"404","message": "Resource not found"}} on every call to a Claude modelStep Test the endpoint | The model was registered with supportedEndpoints: ["/v1/messages"], for example by a custom template or a REST call | Register Foundry Claude models with /anthropic/v1/messages, as the import and claude-code.bicep do. |
| "Backend isn't compatible with all configured MCP versions. The backend reports support for 2025-06-18, but this MCP server requires 2025-11-25…"Step Register the Learn server | More than one version is selected in Supported MCP versions | Select only 2025-06-18. |
RoleAssignmentExistsSteps Deploy the templateDeploy with terraform apply | The gateway's managed identity already has Foundry User on the account, for example from the portal's import | Deploy with grantFoundryUser=false. The step's script sets it for you when it finds the assignment. |
A resource with the ID "…/modelProviders/<name>" already exists - to be managed via Terraform this resource needs to be imported into the State, or the same for an alias or the claude-code keyStep Deploy with terraform apply | The CLI or Bicep track already created those objects on the gateway | Run that track's clean up commands first, or terraform import the object. The deploy block already sets create_provider=false for an existing provider. |
terraform plan wants to destroy azapi_resource.learn[0]Step Deploy the Learn server | The plan ran without -var learn_mcp_server=true, and the variable defaults to false | Pass the -var on every plan once Terraform manages the Learn server. |
anthropic-version: header is requiredStep Test the endpoint | The request has no anthropic-version header | Add -H "anthropic-version: 2023-06-01". Claude Code and the Anthropic SDK send it for you. |
401: Unauthorized. Missing or invalid Entra ID token, or missing Data.All scope.Steps Test the endpointAsk a quick questionTurn Claude Code back on | The key is wrong, suspended, or not active yet, or it's sent in an x-api-key header | Send the key as a bearer token: ANTHROPIC_AUTH_TOKEN for Claude Code. Check the key's state in AccessAPI keys, and wait a minute after you create it. |
| Claude Code hangs, or retries for minutes before it shows an errorSteps Ask a quick questionRun an agent taskCut Claude Code off by suspending the key | The gateway answers 401, which Claude Code retries: the key is in ANTHROPIC_API_KEY instead of ANTHROPIC_AUTH_TOKEN, so Claude Code sends it only in x-api-key, or the key is suspended | Press Ctrl+C, put the key in ANTHROPIC_AUTH_TOKEN, run unset ANTHROPIC_API_KEY, and test the key with Test the endpoint. |
Not logged in · Please run /loginSteps Ask a quick questionRun an agent task | ANTHROPIC_AUTH_TOKEN is empty: AI_GATEWAY_API_KEY wasn't set when you exported it, for example in a new terminal. The Point Claude Code at the gateway step then prints Key: in ANTHROPIC_AUTH_TOKEN (0 characters, not shown) | Read the key again, as your track's Read the key or Create the key for Claude Code step shows, then run Point Claude Code at the gateway again. |
404 with model_not_foundSteps Test the endpointAsk a quick question | The model isn't an alias on the gateway: a typo in an ANTHROPIC_DEFAULT_*_MODEL variable, or a new alias that isn't active yet | Use the aliases that the import creates, such as <foundry-account-name>/claude-opus-5-5, and wait a minute after you publish them. |
Claude Code's debug log shows a 400 on the first request: Unexpected value(s) `advisor-tool-2026-03-01` for the `anthropic-beta` headerStep Ask a quick question | Claude Code tries an experimental beta that the model doesn't accept, then retries without it | Nothing to fix: the answer still arrives. To skip the extra request, set CLAUDE_CODE_DISABLE_EXPERIMENTAL_BETAS=1. |
claude -p doesn't edit the file or run Python, and the result lists permission denialsStep Run an agent task | One-shot mode can't ask for approval | Add --permission-mode acceptEdits --allowedTools "Bash(python3 *)". |
This MCP registration already exists; use mcp update.Step Register the Learn server | The learn server already exists, for example from lab 04 | Reuse it, and go on to Test the MCP endpoint. |
mcp create fails with Operation returned an invalid status 'Bad Gateway'Step Register the Learn server | Another MCP server on the gateway already uses the display name Microsoft Learn, in any letter case, for example lab 04's server under another name | Reuse that server and its endpoint, or pass another --display-name. |
Error: missing required argument 'name'Step Add the server | --header comes before the name and the URL, so Claude Code reads them as more headers | Put --header last, as the step shows. |
Missing environment variables: AI_GATEWAY_API_KEYStep Add the server | The variable isn't set where Claude Code runs | Export AI_GATEWAY_API_KEY, as Point Claude Code at the gateway shows, and run the command again. |
✘ Failed to connect — Server rejected the configured Authorization header (HTTP 401), or ✘ Failed to connect — Dynamic Client Registration rejected (HTTP 404) in older Claude Code versionsSteps Add the serverCut Claude Code off by suspending the key | The gateway rejected the key with 401: the key is wrong, suspended, or not in AI_GATEWAY_API_KEY. Older versions then try to sign in with OAuth, which the gateway doesn't offer | Expected while the key is suspended. Otherwise, check that AI_GATEWAY_API_KEY holds the active claude-code key, and keep the single quotes around the header. |
| Claude Code answers without the Learn tools, or the result lists permission denialsStep Ask a question that needs the docs | One-shot mode can't ask for approval | Add --allowedTools "mcp__ai-gateway-learn__*". |
Every run succeeds, with no 429Step Run Haiku until the token limit | The lower limit isn't active yet, or the requests are mostly cached: Claude Code caches its system prompt and tools, about 13,500 tokens, and cache reads barely count | Wait a minute after you lower the limit, and keep the long-story prompt. |
API Error: Request rejected (429) · Token limit is exceeded. Try again in N seconds.Steps Run Haiku until the token limitAsk a quick questionRun an agent task | The key used its tokens for this minute, and Claude Code doesn't retry a 429 | Wait for the seconds in the message, or raise the token limit. |
Failed to authenticate. API Error: 403 LLM cost quota is exceeded, or Please run /login in a sessionSteps Run Opus until the budgetAsk a quick questionRun an agent task | The key spent its Opus budget for the day, and Claude Code reports a 403 as an authentication error | Don't sign in. Switch with /model sonnet or --model sonnet, raise the budget, or wait until it renews. --fallback-model doesn't cover a 403. |
| The first run is already blockedStep Run Opus until the budget | The earlier steps spent more than $0.05 on Opus today, and the cost limit's id keeps that spend | Expected. To start the count again at $0, give the cost limit a new id. |
Every run succeeds, with no 403Step Run Opus until the budget | Claude Code calls another model: one that you saved with /model sonnet, or set in ANTHROPIC_MODEL | Pass --model opus, as the step shows, or type /model opus in a session. The alias in modelUsage shows which model answered. |
Lab 26: Embeddings
Lab 26: Publish embedding models for search and RAG
| Symptom | Likely cause | Fix |
|---|---|---|
404 with model_not_found: 'text-embedding-3-small' is not a known model alias. right after you create the aliasSteps Embed a sentenceTest in the playgroundEmbed with the OpenAI SDK | A new alias takes a few seconds to reach the gateway | Wait about 5 seconds and try again. |
model_not_found for text-embedding-3-small, and ModelsAliases shows <foundry-account-name>/text-embedding-3-smallSteps Import the embedding modelEmbed a sentence | Create an alias for each model was selected in the import wizard, which names each alias after the Foundry account | Create the alias text-embedding-3-small with AddAlias for existing models, or send the long alias name. |
The model has no Embeddings API format, or --api-format rejects EmbeddingsStep Register the embedding model | --api-format accepts only AnthropicMessages, OpenAIChatCompletions, and ResponsesApi | Use --supported-endpoints /openai/v1/embeddings instead of --api-format. |
AttributeError: 'AAZArgActionOperations' object has no attribute 'strip'Step Register the embedding model | The aigateway extension is older than 1.0.0b33, which fixed model create --foundry-resource-id … --deployment … | Run the installer again, then check extensionVersion with az aigateway version. |
404 with { "statusCode": 404, "message": "Resource not found" }Steps Embed a sentenceEmbed with the OpenAI SDK | The URL has no /default/models, for example {gateway}/openai/v1/embeddings | Use {gateway}/default/models/openai/v1/embeddings. In the SDK, set base_url to {gateway}/default/models/openai/v1. |
404 with unsupported_endpoint: Model 'text-embedding-3-small' does not support the requested endpoint '/default/models/openai/v1/chat/completions'. (or the reverse for gpt-5-mini and /embeddings)Step Call each alias with the wrong API | Each alias accepts only the API of its backend model | Send chat to gpt-5-mini and embeddings to text-embedding-3-small. |
400: Invalid value for 'dimensions' = 4096. Must be less than or equal to 1536.Step Embed a batch with shorter vectors | text-embedding-3-small returns at most 1,536 numbers per vector | Use dimensions from 1 to 1536. |
400: Please submit an `input`.Step Embed a sentence | The body has no input | Send input as a string or an array of strings. |
401: Unauthorized. Missing or invalid Entra ID token, or missing Data.All scope.Steps Embed a sentenceRun the sample | The api-key header is missing or the key is wrong | Reload the key into AI_GATEWAY_API_KEY (see Before you begin). |
The first numbers print as 0,0175…Step Embed a sentence (curl and PowerShell) | -join formats numbers with the culture's decimal comma | Use ConvertTo-Json -Compress as the lab does. |
| The first Generate takes about 10 secondsStep Test in the playground | The first request warms up the route | Wait; later requests take well under a second. |
Every batch returns 200Step Run an ingestion burst | A new token limit takes about 15 to 30 seconds to reach the gateway | Wait 30 seconds after you set the limit and run infra/ingest.py again. |
429 with { "statusCode": 429, "message": "Token limit is exceeded. Try again in 10 seconds." } on every request to the alias, including the wrong-API check, and jq prints null: nullSteps Run an ingestion burstCall each alias with the wrong API | The counter is over the limit, and the gateway applies the token limit before it checks the endpoint. The body isn't in OpenAI's error shape | Wait for Retry-After seconds, or until the minute is over. |
The SDK hangs for a few seconds and then succeeds or raises RateLimitErrorSteps Run an ingestion burstEmbed with the OpenAI SDK | The OpenAI SDK retries 429 twice by default, waiting Retry-After | That's the right behavior for ingestion. infra/ingest.py sets max_retries=0 to show each response. |
Lab 27: Conditions and session affinity
Lab 27: Route alias requests with conditions and session affinity
| Symptom | Likely cause | Fix |
|---|---|---|
The imported model is named gpt-6-1-sol, not gpt-6.1-solStep Import the research model (Portal) | The import replaces dots in the deployment name with hyphens for the resource name, and keeps gpt-6.1-sol as the display name | Select the model by its display name in the portal. In the CLI and Bicep tracks, the model is named gpt-6.1-sol. |
| Models are imported from Foundry accounts you didn't mean to useStep Import the research model (Portal) | Import models preselects every Foundry account in the subscription | Clear every account except your lab's Foundry account before you continue. |
| "Check the expression's parentheses, strings, and comments."Step Create the chat alias with a condition (Portal) | A quote or a parenthesis isn't closed | Fix the syntax. The checker doesn't check that a header exists, so keep the in guard. |
| You ran the step again with other options, and the model keeps its old settingsStep Register the research model (Azure CLI) | az aigateway model create --foundry-resource-id … --deployment … reuses an existing model without changing it | Change the model with az aigateway model update. |
Research requests go to gpt-5-mini, and Read the alias shows always included for gpt-6.1-solStep Create the chat alias with a condition (Azure CLI) | az aigateway model-alias set or update --backend-model replaced the backend models without their when conditions | Run the step's az rest PUT again. Change this alias only with az rest, Bicep, or Terraform. |
when must be a non-empty CEL expression.Steps Create the chat alias with a conditionAdd session affinity (Azure CLI) | The condition is empty | Remove when to include the backend model for every request, or write a condition. |
when must return a Boolean.Steps Create the chat alias with a conditionAdd session affinity (Azure CLI) | The condition returns another type, such as the header value itself | Compare the value, for example request.headers['x-workload'][0] == 'research'. |
undefined field '…'Steps Create the chat alias with a conditionAdd session affinity (Azure CLI) | The expression reads a context property that doesn't exist, such as a typo in request.headers | Use the property names in the CEL context reference. |
Syntax error: mismatched input '<EOF>' …Steps Create the chat alias with a conditionAdd session affinity (Azure CLI) | A quote or a parenthesis isn't closed, often because the shell consumed a quote | Paste the block as is. It wraps the JSON in single quotes and the CEL strings in escaped single quotes. |
key must return a string.Step Add session affinity (Azure CLI) | The session key returns a list or a Boolean, such as request.headers['session-id'] | Read the first value with [0], and return '' when the header is missing. |
sessionBackendPreference and its stage must be configured together.Step Add session affinity (Azure CLI) | sessionBackendPreference is set without "sessionBackendPreference" in selectionOrder, or the reverse | Set both, or remove both. |
selectionOrder requires priority and at most one of each autoModelSelection and sessionBackendPreference stage.Step Add session affinity (Azure CLI) | selectionOrder misses "priority" or repeats a stage | Use ["sessionBackendPreference", "priority"] or ["priority", "sessionBackendPreference"]. |
Requests without the header fail with HTTP 500Steps Create the chat alias with a conditionRead the outputs and the key | The condition reads request.headers['x-workload'][0] without checking that the header exists | Start the condition with 'x-workload' in request.headers &&. |
x-workload: Research goes to gpt-5-miniSteps Create the chat alias with a conditionRead the outputs and the key | Comparisons are case-sensitive | Send the exact value, research. |
The first requests return HTTP 404 model_not_found or use the old routingSteps Create the chat alias with a conditionRead the outputs and the keyEdit the aliasAdd session affinityProvision the affinity mode | The alias change takes a few seconds to reach the gateway | Wait a minute and run the count again. |
HTTP 429 linesSteps Create the chat alias with a conditionRead the outputs and the keyEdit the aliasAdd session affinityProvision the affinity modeCheck that sessions keep their model | A Foundry deployment reached its requests per minute. A capacity of 10 allows about 10 requests per minute, and a raised limit can take minutes to apply | Wait a minute and run the script again, or raise the deployment's capacity. The lab's template uses 50. |
HTTP 503 with an empty body | Every backend model's condition excludes the request | Keep one backend model without a condition, or check that the conditions cover every request. |
| Most sessions map to the same modelSteps Edit the aliasAdd session affinityProvision the affinity mode | The session key picks a backend deterministically, and a few keys can map unevenly | Send more sessions. The split evens out over many keys. |
Lab 28: Auto model selection
Lab 28: Let the gateway choose the model for each request
| Symptom | Likely cause | Fix |
|---|---|---|
The new backend model is named gpt-5-5, not gpt-5.5Step Import gpt-5.5 (Portal) | The import replaces the dots of the deployment name with hyphens, and keeps gpt-5.5 as the display name | Expected. The CLI and Bicep tracks use the same name, gpt-5-5. Its selector model stays gpt-5.5. |
| The list shows deployments from Foundry accounts you didn't mean to useStep Import gpt-5.5 (Portal) | Import models selects every Foundry account it finds | Clear every account except your lab's Foundry account. |
Some models aren't supported for auto model selection.Step Create the auto alias (Portal) | A selected model isn't one that Model Router can choose, such as gpt-6.1-sol | Select gpt-5-mini and gpt-5.5, or check View supported models. |
| Review shows Session affinity EnabledStep Create the auto alias (Portal) | The wizard turns session affinity on by default | Go back to Session affinity and clear Enable session affinity. |
AttributeError: 'AAZArgActionOperations' object has no attribute 'strip'Step Register gpt-5.5 (Azure CLI) | The aigateway extension is older than 1.0.0b33, which fixed model create --foundry-resource-id … --deployment … | Run the installer again, then check extensionVersion with az aigateway version. |
az aigateway model-alias set or update --backend-model fails with selectorModelName must be an unversioned logical Model nameStep Create the auto alias (Azure CLI) | The aigateway extension sends the backend models without their selectorModelName | Change the auto alias with az rest, as the steps do, or with Bicep or Terraform. |
HTTP 404 with 'auto' is not a known model alias.Steps Create the auto aliasRead the outputs and the key | A new alias takes up to a minute to reach every gateway instance | Wait a minute and send the prompts again. |
HTTP 429, retrying in …s linesSteps Create the auto aliasRead the outputs and the keyProvision the Cost objectiveProvision the overrides | A deployment reached its requests per minute. gpt-5.5 has 20 | Nothing: the script waits and retries. To avoid them, raise the deployment's capacity. |
HTTP 503, retrying in …s linesSteps Create the auto aliasRead the outputs and the key | The gateway was busy, often when requests run in parallel | Nothing: the script retries. Send the prompts one at a time, as the script does. |
Short simple prompts, such as What does HTTP stand for?, go to gpt-5.5Steps Create the auto aliasRead the outputs and the key | Balanced weighs quality as well as cost, and a short answer costs little on the larger model. Borderline prompts can change from run to run | Expected. Use the Cost objective to favor gpt-5-mini. |
Some prompts still go to gpt-5.5 with x-ms-model-router-mode: balancedSteps Switch the objectiveProvision the Cost objective | The alias change hadn't reached every gateway instance | Wait a minute and run the script again. |
HTTP 403 request_override_not_allowed: The Alias does not permit every supplied auto_model_selection field.Steps Allow per-request overridesProvision the overrides | The alias doesn't allow that override yet, or the change that allows it hasn't reached every gateway instance | Allow the override (Allow per-request overrides), then wait a minute. |
After you change only the objective, Caller overrides reads noneStep Allow per-request overrides (Azure CLI) | A PATCH replaces the whole autoModelSelection object | Send objective and allowedRequestOverrides together, as the step does. |
HTTP 400: auto_model_selection.models must contain unique names from this Alias's configured selector Model pool.Steps Allow per-request overridesProvision the overrides | models holds a backend model name, such as gpt-5-5, a model that isn't in the alias, or the same name twice | Use the selector model names, gpt-5-mini and gpt-5.5. |
HTTP 400: auto_model_selection.objective must be balanced, cost or quality.Steps Allow per-request overridesProvision the overrides | The objective is misspelled | Send balanced, cost, or quality. |
HTTP 400: auto_model_selection accepts only unique objective and models fields.Steps Allow per-request overridesProvision the overrides | The body has another field, such as model, or repeats one | Send only objective and models. |
The response has no x-ms-model-router-mode headerSteps Allow per-request overridesProvision the overrides | models narrowed the pool to one model, so there was nothing for Model Router to choose | Expected. x-ms-served-model still names the model. |
balanced shows gpt-5-mini-2025-08-07Step Send the hard prompt with each objective | Balanced picks per prompt, and the choice for one prompt can vary | Expected. cost and quality are stable. |
Lab 29: Model failover
Lab 29: Fail over to a backup model when a backend fails
| Symptom | Likely cause | Fix |
|---|---|---|
ERROR: The model already exists. Use az aigateway model update to change its configuration.Step Register the two models | The backend model is already in the gateway, for example from lab 22 or 27, or from a portal import | Skip that command. az aigateway model create doesn't update an existing model. |
No 429s in the first burst, and every request shows gpt-5-mini-2025-08-07Step Register the two models | modelName in --deployment-config names gpt-5-mini instead of gpt-5-mini-tiny. The gateway calls the deployment that modelName names, so the requests go to the large deployment | Delete the backend model and register it again with modelName:$PRIMARY. |
Backend models shows gpt-6-1-sol instead of gpt-6.1-solStep Import the two models | The portal import replaces the dot in the deployment name | Nothing to fix. Select gpt-6-1-sol where the lab says gpt-6.1-sol. Read the alias shows the same name. |
| No Maximum retries per request field in Failure handlingStep Add a failure rule (Portal) | The Request retry limit appears only once the alias has a rule | Add the rule first, then check that the field is 3. |
Some requests return HTTP 503 with an empty body and no x-ms-served-modelSteps Add a failure rule (Azure CLI)Check that the primary comes back | The alias has the failure rule, but maxRetryAttempts is 0, the API default, so the gateway can't send the request to the backup | Send "maxRetryAttempts": 3 together with the rule, as in the lab, and read the alias again. |
Failure rules: none, and 429 with Retry-After: 30 and Your requests to gpt-5-mini for gpt-5-mini-tiny in <region> have exceeded request rate limit.Steps Read the aliasCheck that the primary comes back | The rule didn't reach the alias. In the portal, the edit wasn't saved. On the Bicep + azd track, FAILOVER isn't on | Open the alias and select Save, or run azd env set FAILOVER on and azd provision. |
The first requests show gpt-6.1-sol-2026-09-29, not gpt-5-mini-2025-08-07Step Check that the primary comes back | The primary is still excluded, for 30 seconds after its last 429, or still out of capacity for the minute | Wait a minute, then send the requests again. |
Some requests still show 429 right after you turn on failoverSteps Add a failure ruleProvision with failover onApply with failover on | The gateway takes a few seconds to apply an alias change, and the primary's Retry-After: 30 window from the previous burst may still be open | Wait a minute after the change, then send the burst again. |
Lab 30: Agent hooks
Lab 30: Govern MCP tool calls with agent hooks
| Symptom | Likely cause | Fix |
|---|---|---|
The first code upload fails with BlobUploadFailedException: Failed to upload blob to storage account: Resource temporarily unavailable (<storage>.blob.core.windows.net:443)Steps Deploy the codeDeploy with azd upDeploy with terraform apply | A new Flex Consumption app that reaches its storage through a private endpoint can't upload until it restarts once | Run az functionapp restart -g <resource-group> -n <function-app> and deploy again. On the Bicep + azd track, the postprovision hook restarts the app; if it still fails, run azd deploy. |
agentHooks contains an invalid lifecycle destination configuration.Step Write the policy | The policy has no authentication, or the endpoint URL isn't https:// | Set endpoint.url to the function's HTTPS URL and add authentication.managedIdentity.resource with the hook's application ID URI |
agentHooks.hooks contains a lifecycle point that is not supported on ToolServer.Step Write the policy | The policy has preModelCall or postModelCall. MCP servers support only the tool-call points. | Keep only preToolCall and postToolCall |
Invalid field 'authentication' from az aigateway mcp update --policiesStep Write the policy | The aigateway extension is older than 1.0.0b34, which doesn't know the agent hooks shape of API version 2026-11-01 | Run the installer again, then run the step again. |
The hook's header value reads "***"Step Read the policy back | The aigateway extension masks header values in its output | Expected: the gateway still sends the real value, as the decision log in Verify shows. Don't copy a masked value back into --policies. |
| After you edit the policy in the portal, the after-tool-call hook is goneSteps Configure the hookRead the policy back | The portal editor supports one intervention point. On a policy with both, it reads "This policy has both intervention points. Choose one to continue." and keeps only the point that you choose. | Edit policies with both points with the Azure CLI or Bicep. If the portal already dropped one, run the Write the policy step again. |
Every call returns isError: true with "Tool call blocked by a policy before execution", and the decision log has no linesStep Call an allowed tool | The gateway can't get a verdict, and failClosed treats that as a deny. App Service Authentication rejects the token with 401 (the policy's managedIdentity.resource isn't the app's application ID URI, or the gateway's identity isn't allowed), the app is stopped, or the endpoint URL is wrong | Compare managedIdentity.resource with HOOK_AUDIENCE and the endpoint with HOOK_URL, start the function app, and check its Authentication settings. |
The harmless search still returns results right after az functionapp stopStep See the policy fail closed | A Flex Consumption app takes a few seconds to stop | Wait 30 seconds after the stop, then call again |
| The query returns no linesStep Read the decision log | Application Insights takes two to five minutes to ingest the traces | Wait a few minutes and run the query again |
An app registration named aigw-lab30-hook-<suffix> remains after azd downStep Delete everything azd created | The predown hook couldn't delete it, for example because the azd environment lost its outputs or you aren't an owner of the app | Run az ad app delete --id <client-id>, then purge it from Deleted applications in Microsoft Entra ID |
A tool call fails with initialize failed with HTTP 503: {"error":"App port is not ready"}, minutes after the deploymentSteps Call an allowed toolCall a blocked tool | The MCP server's runtime wasn't warm yet | Wait a few seconds and run the call again. |
Lab 31: Jev decisions
Lab 31: Publish Jev decision models for typed answers
| Symptom | Likely cause | Fix |
|---|---|---|
Model-provider authentication requires exactly one API-key header.Step Add the TypeSafe provider | The provider has two authentication headers, for example Authorization and api-key | Keep one header: Authorization with the value Bearer <TypeSafe key>. |
The output shows "kind": null, and endpoint is an object with urlStep Add the TypeSafe provider | The query is written for the aigateway extension 1.0.0b33 or earlier. From 1.0.0b34, the extension prints the names of API version 2026-11-01 | Query properties.modelProviderType and properties.custom.endpoint.url, as the step does. |
The reserved System One endpoint must be typesafe/v1/systemone without a leading '/'.Step Publish the Jev model | --supported-endpoints /typesafe/v1/systemone has a leading slash | Use --supported-endpoints typesafe/v1/systemone. |
Active-priority models must share at least one supported endpoint.Step Create the alias | The alias mixes jev-latest with a chat or embedding model, at any priority | Give decision models their own alias. An alias can route across several Jev models. |
404 with { "statusCode": 404, "message": "Resource not found" }, or model_not_found for ticket-triage, right after you create the aliasSteps Run decisions in the playgroundTriage a ticketRun the sample | New resources take up to a minute to reach the runtime | Wait a few seconds and try again. |
| The first Run decisions takes about 5 secondsStep Run decisions in the playground | The first call to a new model warms up the route | Wait. Later calls take about half a second. |
400: Specify either an API key or an Authorization header, not both.Step Triage a ticket | The request has both Authorization: Bearer and api-key | Send the gateway key in one header. |
401: Unauthorized. Missing or invalid Entra ID token, or missing Data.All scope. (TypeSafeAuthenticationError in the SDK)Steps Triage a ticketRun the sample | The gateway key is missing or wrong, even though the message mentions only Entra, or the TypeSafe key was sent instead of the gateway key | Reload the gateway key into AI_GATEWAY_API_KEY. Apps never use the TypeSafe key. |
401: {"detail":{"error_type":"authentication_error","message":"Cannot authenticate with the server. Please check your API key and try again."}}Step Triage a ticket | The gateway accepted your key, but TypeSafe rejected the key stored in the provider | Check the key in TypeSafe's console and set the right key on the typesafe provider. |
404 with model_not_found: 'jev-latest' is not a known model alias.Step Triage a ticket | model is the backend model's name | Send the alias, ticket-triage, in model. |
400 with invalid_request: A model alias is required in the request body.Step Triage a ticket | The body has no model | Add "model": "ticket-triage". |
400: {"detail":{"error_type":"api_usage_error","message":"Invalid request."}}Step Triage a ticket | TypeSafe rejected a question, for example an unknown type | Use noul, choice, or score, with criteria for choice and score, as in infra/ticket.json. |
415 with SystemOneEncodingUnsupported: System One requires uncompressed JSON. Disable SDK request compression.Steps Point the TypeSafe SDK at the gatewayRun the sample | On Python 3.14 or with typesafe-sdk[zstd], the SDK compresses bodies of 1 KB or more with zstd | Pass compression=False to TypeSafeClient. |
requestRateLimit.callsPerPeriod must be greater than 0.Step Set the request rate limit | The aigateway extension is older than 1.0.0b33, which maps count and period | Run the installer again, then run the step again. |
Set your TypeSafe key in TF_VAR_jev_api_key before terraform plan.Steps Deploy with terraform applyPreview a change to the limit | TF_VAR_jev_api_key isn't set in this shell, for example in a new terminal | Run the hidden prompt from Deploy with terraform apply again, then plan. terraform destroy doesn't need the key. |
The ephemeral input variable "jev_api_key" was set during the plan phase, and so must also be set during the apply phase.Step Deploy with terraform apply | terraform apply tfplan ran in a shell without TF_VAR_jev_api_key | Set the key again in that shell, or plan and apply in the same shell. |
Unsupported Terraform Core versionStep Deploy with terraform apply | Terraform is older than 1.11, which the write-only sensitive_body argument needs | Install Terraform 1.11 or later. |
429 before request 6, or on every requestStep Send a burst of decisions | Earlier calls in the same minute count against the limit | Wait 60 seconds and run the burst again. |
Every request returns 200Step Send a burst of decisions | With azd, DECISIONS_PER_MINUTE is higher than 7, for example 10 | Run azd env set DECISIONS_PER_MINUTE 5 and azd provision. |
Six requests pass with a limit of 5Step Send a burst of decisions | The gateway admitted one request more than count in this lab's runs | Expected. Size production limits with this in mind. |
Lab 32: Foundry gateway connection
Lab 32: Use gateway models in Microsoft Foundry projects
| Symptom | Likely cause | Fix |
|---|---|---|
| The Foundry portal shows No accessStep Create a Foundry project | The Foundry portal is signed in to a different directory (tenant) than your subscription | Select your avatar › Switch, and pick the directory of your subscription. |
| The app team's resource group isn't in the Resource group listStep Create a Foundry project | The Create a project dialog lists only resource groups that exist, and can't create one | Create the resource group first, as in Create the app team's resource group. |
| ManageGateways doesn't list your gatewayStep Open the gateway in Foundry | Subscription at the top of the page is a different subscription than the gateway's | Pick the gateway's subscription in Subscription. |
| Configure models doesn't list your aliasStep Pick the models | Foundry hasn't loaded the gateway's new models yet, or the alias doesn't exist | Check that the gateway's Models page lists the alias in the AI Gateway portal, then select Refresh above the table in Configure models. |
| Project detailsConnections shows the connection with an API key instead of Project Managed IdentityStep Choose managed identity | The wizard's Authentication step starts on API key | Delete the connection and add it again with Managed identity. |
embedded schema validation failed: … properties.authType's value ProjectManagedIdentity is invalid.Step Deploy with terraform apply | schema_validation_enabled = true on azapi_resource.connection: the azapi provider's embedded schema predates ProjectManagedIdentity | Keep schema_validation_enabled = false on the connection. |
403 with Access to model is denied - Access denied. You do not have permission to access the requested resource. Ask an administrator to grant the required Azure role. (ApiSamplingErrorUnauthorized)Steps Chat in the playgroundCall the model through the projectCall the model and an agent | The project's managed identity doesn't hold AI Gateway Models User on the gateway's default workspace, or the new role hasn't reached the gateway yet | Assign the role, as in Give the project access to the gateway's models, then wait a minute and try again. The Foundry portal doesn't assign it for you. |
404 with ResourceNotFound: Subdomain does not map to a resource., or Project not foundSteps Chat in the playgroundCall the model through the projectCall the model and an agent | The Foundry resource is new, or reuses the name of a Foundry resource that you purged recently, and its endpoint isn't routable yet. Some routes can lag behind others: in this lab's runs, /responses worked while Chat Completions and deleting or listing agents still failed after 45 minutes | Wait a few minutes and try again, or create the Foundry resource with a name that you haven't used. |
| Save as agent is turned offStep Chat in the playground | The playground can't save an agent on a connected model | Create the agent from AgentsNew agentBuild an agent. |
404 with DeploymentNotFound: The API deployment for this resource does not exist. If you created the deployment within the last 5 minutes, please wait a moment and try again.Step Call Chat Completions through the project | Foundry serves connected models through the Responses API only. Waiting doesn't help | Expected. Call /openai/v1/responses with the model's Foundry name, <gateway>-model-gateway/<alias>. |
The SDK's burst takes several seconds per request, and fewer requests than expected show 429Step Reach the limit through the project | The OpenAI client retries a 429 twice before it raises the error | Expected. Run python infra/call_model.py --burst 4, which turns the retries off with max_retries=0. |
CannotDeleteResource when you delete the Foundry resourceStep Delete the app team's project | The Foundry resource still has a project | Delete the project first, then the Foundry resource. |
Lab 33: Copilot on-behalf-of
Lab 33: Connect Microsoft 365 Copilot to MCP tools as the signed-in user
| Symptom | Likely cause | Fix |
|---|---|---|
The deployment fails with an error that starts with Another object with the same value for property uniqueNameSteps Deploy the backendDeploy with azd upDeploy with terraform apply | An app registration of an earlier run with the same base name is still in Deleted applications | Purge it from Deleted applications in Microsoft Entra ID, as Clean up does, or deploy with a new base name. |
A role assignment fails with PrincipalNotFoundSteps Deploy the backendGrant yourself the Tools User roleDeploy with azd upDeploy with terraform apply | The new managed identity or service principal hasn't reached every Microsoft Entra ID replica yet | Wait a minute and run the step again. |
ERROR: Operation returned an invalid status 'Bad Gateway'Step Register the MCP server | Another MCP server on the gateway already has the display name OBO demo. Display names must be unique on a gateway | Delete the other server, or pass another --display-name. |
The portal says "Use explicit delegated scopes rather than .default."Step Validate the backend | The backend scope ends in /.default | Enter the full scope, api://<backend app ID>/user_impersonation. |
Every tool call fails with 502Steps Test in the catalogRun the test scriptAsk who the backend sees | The token exchange app's identifier URI isn't api://<client ID>, which is how the gateway addresses it | Run az ad app update --id <token exchange app ID> --identifier-uris "api://<token exchange app ID>". The lab's templates set it for you. |
502 or the JSON-RPC error -32603, or 400 with -32602 when the server supports only 2026-07-28Steps Test in the catalogRun the test script | The server's supported MCP versions don't include the version that the client negotiates | Support 2025-11-25, 2025-06-18, and 2025-03-26, as Register the MCP server does. |
403 on the handshakeSteps Test in the catalogRun the test script | You don't hold the AI Gateway Tools User role on the MCP server, or the new assignment hasn't applied yet | Run Grant yourself the Tools User role, then wait a few minutes and try again. |
| "Selected user account does not exist in tenant"Step Add the connector | The browser is signed in with an account of another tenant | Sign in with an account of the tenant that holds the connector app, for example in an InPrivate window. |
| "The sign-in window was blocked by the browser. Please enable pop-ups for this site and retry authentication to complete setup."Step Add the connector | The browser blocks the admin center's pop-ups | Allow pop-ups for the Microsoft 365 admin center, then select Authorize again. |
| AI Gateway OBO demo isn't Ready yetStep Check the connection and its tools | Microsoft 365 takes up to 30 minutes to prepare a new connector | Wait, then refresh Your Connections. |
| Every tool shows as Write, and Copilot asks you to approve each callSteps Check the connection and its toolsAsk who the backend sees | The gateway doesn't pass the tools' readOnlyHint annotations on to the connector yet | Expected. Select Approve. |
| In Copilot Chat, the call never finishes, and Tasks shows it at Needs your inputStep Ask who the backend sees | Chat doesn't show the approval card of a Write tool | Use Cowork, as the step does. |
| The query returns no linesStep Read the backend's log | Log Analytics takes a few minutes to ingest the container's logs | Wait a few minutes and run the query again. |
terraform destroy runs for about 25 minutesStep Delete everything Terraform created | Azure takes that long to delete the Container Apps environment | Expected. Wait for Destroy complete!. |