The first product team wants to use gpt-5-mini, which already runs in your Microsoft Foundry account. You
don't want to hand every team the Foundry endpoint and a provider key. In this lab you publish the deployment
through your gateway, give the team a runtime API key of its own, and make the first call from the playground,
a shell, and code. The gateway authenticates to Foundry with its own managed identity, so the app never sees a
Foundry credential.
Before you begin
One-time setupThe Setup guide tasks that this lab uses. Mark each one done once, and the labs stop listing it.
- Choose where to run commandsEvery track except the portal
- Sign in to AzureOn your own machine. Cloud Shell is already signed in.
- Install the
aigatewayextensionThe Azure CLI tracks and the Verify steps - Clone the labs repoThe Bicep + azd and Terraform tracks, the notebooks, and the code samples
You've done the one-time setup that this lab uses.
On the Bicep + azd and Terraform tracks, azd up or terraform apply creates its own gateway, Foundry
account, model, and key, so you can skip this section. You need only the one-time setup above, plus the language
runtime for the code track if you follow it.
- Complete Provision your AI Gateway. You need the gateway, its resource group, and its runtime URL.
- A model deployment in a Microsoft Foundry (Azure AI Services) or Azure OpenAI account, for example
gpt-5-mini. If you don't have one, deploy the optional template described below. - Permission to grant the gateway access to the account: Owner, User Access Administrator, or Role Based Access Control Administrator on the Foundry account. You also need to change the gateway, for example Contributor on its resource group.
- For the code track: Python 3.9 or later, Node.js 22.18 or later, or the .NET 8 SDK.
- Run the scripts and samples from the lab's folder,
~/aigateway/labs/02-first-model-call.
Set these variables once. Every track and the Verify section reuse them.
# Name your gateway and its resource group, the Foundry account and its deployment, and the provider to create
export RG="<resource-group>"
export GATEWAY="<gateway>"
export FOUNDRY_RG="<foundry-resource-group>" # resource group of the Foundry account
export FOUNDRY_ACCOUNT="<foundry-account-name>" # the account name, not a project or deployment name
export DEPLOYMENT="gpt-5-mini" # the model deployment to publish
export PROVIDER="foundry-models" # CLI track: the gateway provider to create
echo "📝 Gateway $GATEWAY in $RG, deployment $DEPLOYMENT in Foundry account $FOUNDRY_ACCOUNT"
# Read the gateway's resource ID, and build its runtime URL from its host name
export GATEWAY_ID="$(az aigateway show -g "$RG" -n "$GATEWAY" --query id -o tsv)"
export AI_GATEWAY_URL="https://$(az aigateway show -g "$RG" -n "$GATEWAY" --query properties.frontend.defaultHostname -o tsv)"
echo "🆔 Gateway ID: $GATEWAY_ID"
echo "🌐 Runtime URL: $AI_GATEWAY_URL"# Name your gateway and its resource group, the Foundry account and its deployment, and the provider to create
$RG = "<resource-group>"
$GATEWAY = "<gateway>"
$FOUNDRY_RG = "<foundry-resource-group>" # resource group of the Foundry account
$FOUNDRY_ACCOUNT = "<foundry-account-name>" # the account name, not a project or deployment name
$DEPLOYMENT = "gpt-5-mini" # the model deployment to publish
$PROVIDER = "foundry-models" # CLI track: the gateway provider to create
Write-Host "📝 Gateway $GATEWAY in $RG, deployment $DEPLOYMENT in Foundry account $FOUNDRY_ACCOUNT"
# Read the gateway's resource ID, and build its runtime URL from its host name
$GATEWAY_ID = az aigateway show -g $RG -n $GATEWAY --query id -o tsv
$env:AI_GATEWAY_URL = "https://$(az aigateway show -g $RG -n $GATEWAY --query properties.frontend.defaultHostname -o tsv)"
Write-Host "🆔 Gateway ID: $GATEWAY_ID"
Write-Host "🌐 Runtime URL: $env:AI_GATEWAY_URL"No Foundry deployment yet? Create one with Bicep.
infra/foundry.bicep creates an Azure AI Services account with a gpt-5-mini
GlobalStandard deployment, mirroring the reference sample. Pick a region where the model is available. Model
inference is billed per use.
# Create the resource group for the Foundry account
az group create --name "$FOUNDRY_RG" --location eastus2 --output none &&
echo "📦 Resource group $FOUNDRY_RG is ready in eastus2"
# Deploy the template: the Foundry account and its gpt-5-mini deployment, then read the deployment's resource ID
DEPLOYMENT_ID="$(az deployment group create -g "$FOUNDRY_RG" \
--template-file infra/foundry.bicep \
--parameters accountName="$FOUNDRY_ACCOUNT" deploymentName="$DEPLOYMENT" \
--query properties.outputs.deploymentId.value -o tsv)" &&
echo "✨ Model deployment $DEPLOYMENT is ready: $DEPLOYMENT_ID"How it works
Publishing a model has three parts. A model provider tells the gateway where the Foundry account is and
how to authenticate to it. A backend model registers one deployment under that provider. A callable
alias is the stable name that clients send in the request model field. Importing from Foundry creates all
three for you.
Clients call one runtime URL with a gateway API key. The gateway validates the key before anything else, runs the policies that apply, swaps the caller's key for its own managed identity token, and forwards the request unchanged to Foundry. The response comes back as is, and the gateway records token usage for monitoring.
| Step | What happens | Where you set it |
|---|---|---|
| Inbound authentication | An active runtime key as Authorization: Bearer <key>, which is what the OpenAI SDKs send, or in an api-key header. Without a valid key the call fails with 401, and with both headers it fails with 400 | AccessAPI keys |
| Routing | /default/models/openai/v1/chat/completions or /responses, and model must equal an alias | Model details and its Aliases tab |
| Policies | Token limits, cost limits, and content safety run before the backend call | Policies (labs 06–09) |
| Backend authentication | The gateway's managed identity, with the Foundry User role on the account | Model provider |
| Telemetry | Tokens, latency, and the API key that made the call | Monitoring (lab 13) |
Publish the model and create a key
Pick one track. Each track publishes the deployment, gives you a callable alias, and creates a runtime API key. The Bicep + azd and Terraform tracks also create their own gateway and Foundry deployment, so they don't need lab 01's gateway.
Import the deployment from Foundry
- Open the AI Gateway portal and select your gateway in the top bar.
- Select ModelsAddModels from Microsoft Foundry. (On Home, Add from Microsoft Foundry in the Set up your gateway guide starts the same import.)
- In Subscriptions and Resource groups, select the subscription and resource group of your Foundry account. The page discovers the deployments automatically. Deployments that are already in this gateway can't be selected again.
- Select the
gpt-5-minideployment and keep Managed identity as the Authentication method. - Under Aliases, keep Create an alias for each model selected, then select Add models. When the banner Models and aliases imported shows, select Close.
The gateway calls Foundry with its system-assigned managed identity, which needs the Foundry User role on
the account. If your first call returns 403 from the backend, assign the role yourself and wait a few
minutes for it to take effect:
# Read the principal ID of the gateway's managed identity, and the Foundry account's resource ID
PRINCIPAL_ID="$(az aigateway show -g "$RG" -n "$GATEWAY" --query identity.principalId -o tsv)"
FOUNDRY_ID="$(az cognitiveservices account show -g "$FOUNDRY_RG" -n "$FOUNDRY_ACCOUNT" --query id -o tsv)"
echo "👤 Gateway identity: $PRINCIPAL_ID"
# Give the gateway's identity the Foundry User role on the account, so that it can call the model
az role assignment create --assignee-object-id "$PRINCIPAL_ID" --assignee-principal-type ServicePrincipal \
--role "53ca6127-db72-4b80-b1b0-d745d6d5456d" --scope "$FOUNDRY_ID" --output none &&
echo "🔐 The gateway's identity has the Foundry User role on $FOUNDRY_ACCOUNT"Copy the callable alias
On Models, the Backend models tab lists gpt-5-mini with its alias. Select the model: its Overview
lists the Endpoints that it supports, such as Chat completions and Responses, and its Aliases tab
lists the aliases that route to it. Copy the alias exactly as shown, then save it in your shell. Portal imports name
it after the provider and the deployment, for example <foundry-account-name>/gpt-5-mini.
# Save the alias that apps send in the model field
export AI_GATEWAY_MODEL="<model-name>" # the alias from the Aliases tab
echo "🔖 Model alias: $AI_GATEWAY_MODEL"# Save the alias that apps send in the model field
$env:AI_GATEWAY_MODEL = "<model-name>" # the alias from the Aliases tab
Write-Host "🔖 Model alias: $env:AI_GATEWAY_MODEL"Create a runtime API key
- Select Access, then the API keys tab, then Create API key.
- In Key name, enter a name for the app that will use it, for example
app-first-call, and select Create. - In API key created, select Copy & done to copy the key. You can view, rotate, or delete it later in AccessAPI keys.
- Paste it into your shell without echoing it or writing it to a file:
# Paste the key without showing it, and export it for the calls and samples that follow
printf "AI Gateway API key: " && read -rs AI_GATEWAY_API_KEY && export AI_GATEWAY_API_KEY && echo
echo "🔑 Key: in AI_GATEWAY_API_KEY (${#AI_GATEWAY_API_KEY} characters, not shown)"# Paste the key without showing it, into the environment variable that the calls and samples read
$env:AI_GATEWAY_API_KEY = Read-Host "AI Gateway API key" -MaskInput
Write-Host "🔑 Key: in AI_GATEWAY_API_KEY ($($env:AI_GATEWAY_API_KEY.Length) characters, not shown)"Models lists the Foundry deployment, its Aliases tab shows an alias, and AccessAPI keys shows
app-first-call.
Look up the Foundry account
Use the account name and resource group to look up its resource ID and endpoint. No need to copy a long ID from the portal.
# Read the Foundry account's resource ID and endpoint, which the provider and the model use
export FOUNDRY_ID="$(az cognitiveservices account show -g "$FOUNDRY_RG" -n "$FOUNDRY_ACCOUNT" --query id -o tsv)"
FOUNDRY_ENDPOINT="$(az cognitiveservices account show -g "$FOUNDRY_RG" -n "$FOUNDRY_ACCOUNT" --query properties.endpoint -o tsv)"
export FOUNDRY_ENDPOINT="${FOUNDRY_ENDPOINT%/}/" # the provider expects a trailing slash
echo "🆔 Foundry account ID: $FOUNDRY_ID"
echo "🌐 Foundry endpoint: $FOUNDRY_ENDPOINT"# Read the Foundry account's resource ID and endpoint, with the trailing slash that the provider expects
$FOUNDRY_ID = az cognitiveservices account show -g $FOUNDRY_RG -n $FOUNDRY_ACCOUNT --query id -o tsv
$FOUNDRY_ENDPOINT = (az cognitiveservices account show -g $FOUNDRY_RG -n $FOUNDRY_ACCOUNT --query properties.endpoint -o tsv).TrimEnd('/') + '/'
Write-Host "🆔 Foundry account ID: $FOUNDRY_ID"
Write-Host "🌐 Foundry endpoint: $FOUNDRY_ENDPOINT"Grant the gateway access to the account
The portal import does three things for you: it grants the gateway's identity access to the account, creates the provider, and registers the deployments. From the command line, you do each step yourself. First, give the gateway's system-assigned identity the Foundry User role on the account.
# Read the principal ID of the gateway's system-assigned managed identity
PRINCIPAL_ID="$(az aigateway show -g "$RG" -n "$GATEWAY" --query identity.principalId -o tsv)"
echo "👤 Gateway identity: $PRINCIPAL_ID"
# Give the identity the Foundry User role on the account, so that the gateway can call its deployments
az role assignment create --assignee-object-id "$PRINCIPAL_ID" --assignee-principal-type ServicePrincipal \
--role "53ca6127-db72-4b80-b1b0-d745d6d5456d" --scope "$FOUNDRY_ID" --query id -o tsv &&
echo "🔐 The gateway's identity has the Foundry User role on $FOUNDRY_ACCOUNT"# Read the principal ID of the gateway's system-assigned managed identity
$PRINCIPAL_ID = az aigateway show -g $RG -n $GATEWAY --query identity.principalId -o tsv
Write-Host "👤 Gateway identity: $PRINCIPAL_ID"
# Give the identity the Foundry User role on the account, so that the gateway can call its deployments
az role assignment create --assignee-object-id $PRINCIPAL_ID --assignee-principal-type ServicePrincipal `
--role "53ca6127-db72-4b80-b1b0-d745d6d5456d" --scope $FOUNDRY_ID --query id -o tsv
if ($?) { Write-Host "🔐 The gateway's identity has the Foundry User role on $FOUNDRY_ACCOUNT" }Create the Foundry provider
The provider tells the gateway where the account is and to authenticate with its managed identity.
# Create a Foundry provider that points at the account and signs in with the gateway's managed identity
az aigateway model-provider create -g "$RG" --gateway-name "$GATEWAY" -n "$PROVIDER" \
--kind Foundry --display-name "Foundry models" \
--foundry "{endpoint:'$FOUNDRY_ENDPOINT',resourceIds:[$FOUNDRY_ID],authentication:{kind:ManagedIdentity,managedIdentity:{resource:'https://cognitiveservices.azure.com/'}}}" \
--query "{name:name, kind:properties.modelProviderType, endpoint:properties.foundry.endpoint.url}" &&
echo "🔌 Model provider $PROVIDER points at $FOUNDRY_ACCOUNT"# Create a Foundry provider that points at the account and signs in with the gateway's managed identity
az aigateway model-provider create -g $RG --gateway-name $GATEWAY -n $PROVIDER `
--kind Foundry --display-name "Foundry models" `
--foundry "{endpoint:'$FOUNDRY_ENDPOINT',resourceIds:[$FOUNDRY_ID],authentication:{kind:ManagedIdentity,managedIdentity:{resource:'https://cognitiveservices.azure.com/'}}}" `
--query "{name:name, kind:properties.modelProviderType, endpoint:properties.foundry.endpoint.url}"
if ($?) { Write-Host "🔌 Model provider $PROVIDER points at $FOUNDRY_ACCOUNT" }Register the deployment
A backend model registers one deployment under the provider. Name it after the deployment.
# Register the deployment as a backend model under the provider, with the endpoints that apps can call
az aigateway model create -g "$RG" --gateway-name "$GATEWAY" --provider-name "$PROVIDER" -n "$DEPLOYMENT" \
--display-name "$DEPLOYMENT" --api-format OpenAIChatCompletions \
--supported-endpoints /openai/v1/chat/completions /openai/v1/responses \
--deployment-config "{resourceId:$FOUNDRY_ID/deployments/$DEPLOYMENT,modelName:$DEPLOYMENT}" \
--query "{name:name, deployment:properties.deployment.modelName}" &&
echo "✨ Model $DEPLOYMENT is registered under $PROVIDER"# Register the deployment as a backend model under the provider, with the endpoints that apps can call
az aigateway model create -g $RG --gateway-name $GATEWAY --provider-name $PROVIDER -n $DEPLOYMENT `
--display-name $DEPLOYMENT --api-format OpenAIChatCompletions `
--supported-endpoints /openai/v1/chat/completions /openai/v1/responses `
--deployment-config "{resourceId:$FOUNDRY_ID/deployments/$DEPLOYMENT,modelName:$DEPLOYMENT}" `
--query "{name:name, deployment:properties.deployment.modelName}"
if ($?) { Write-Host "✨ Model $DEPLOYMENT is registered under $PROVIDER" }--deployment-config uses the Azure CLI shorthand syntax, so the same value works in Bash and PowerShell.
To publish more deployments, repeat this step and the next one for each name that
az cognitiveservices account deployment list -g "$FOUNDRY_RG" -n "$FOUNDRY_ACCOUNT" --query "[].name" -o tsv
returns.
az aigateway model-provider import -g "$RG" --gateway-name "$GATEWAY" -n "$PROVIDER" --foundry-resource-id "$FOUNDRY_ID"
does the last three steps in one command for every deployment in the account, like the portal import: it grants
the gateway's identity Foundry User on the account, creates the provider, and registers each deployment. You
can run it again safely. It doesn't publish an alias, so you still run the next step. The
quick start uses it. This lab creates each piece itself so you see how they fit together.
Publish an alias
Clients send an alias in the request model field, not the backend model name. Publish one named after the
deployment that points at the model you registered.
# Name the alias after the deployment, and build the resource ID of the model that it points at
export AI_GATEWAY_MODEL="$DEPLOYMENT"
MODEL_ID="$GATEWAY_ID/workspaces/default/modelProviders/$PROVIDER/models/$DEPLOYMENT"
# Publish the alias, or update it if it exists, and print its name and the model that it points at
az aigateway model-alias set -g "$RG" --gateway-name "$GATEWAY" -n "$AI_GATEWAY_MODEL" \
--backend-model resource-id="$MODEL_ID" \
--query "{alias:properties.aliasName, target:properties.backendModels[0].resourceId}" &&
echo "🔖 Alias $AI_GATEWAY_MODEL is ready"# Name the alias after the deployment, and build the resource ID of the model that it points at
$env:AI_GATEWAY_MODEL = $DEPLOYMENT
$MODEL_ID = "$GATEWAY_ID/workspaces/default/modelProviders/$PROVIDER/models/$DEPLOYMENT"
# Publish the alias, or update it if it exists, and print its name and the model that it points at
az aigateway model-alias set -g $RG --gateway-name $GATEWAY -n $env:AI_GATEWAY_MODEL `
--backend-model resource-id="$MODEL_ID" `
--query "{alias:properties.aliasName, target:properties.backendModels[0].resourceId}"
if ($?) { Write-Host "🔖 Alias $env:AI_GATEWAY_MODEL is ready" }az aigateway model-alias set creates the alias, or updates it if it exists, so you can run the step again.
Create a runtime API key
api-key create returns metadata only. Read the generated value with list-secrets straight into a variable so
it never appears on screen.
# Create a runtime API key for your app, which prints its name and state but not its value
az aigateway api-key create -g "$RG" --gateway-name "$GATEWAY" -n app-first-call \
--query "{name:name, state:properties.state}" &&
echo "🔑 API key app-first-call is ready"
# Read the key's value straight into an environment variable, so that it never shows on screen
AI_GATEWAY_API_KEY="$(az aigateway api-key list-secrets -g "$RG" --gateway-name "$GATEWAY" -n app-first-call \
--query primaryKey -o tsv)" && export AI_GATEWAY_API_KEY
echo "🔑 Key: in AI_GATEWAY_API_KEY (${#AI_GATEWAY_API_KEY} characters, not shown)"# Create a runtime API key for your app, which prints its name and state but not its value
az aigateway api-key create -g $RG --gateway-name $GATEWAY -n app-first-call `
--query "{name:name, state:properties.state}"
if ($?) { Write-Host "🔑 API key app-first-call is ready" }
# Read the key's value straight into an environment variable, so that it never shows on screen
$env:AI_GATEWAY_API_KEY = az aigateway api-key list-secrets -g $RG --gateway-name $GATEWAY -n app-first-call `
--query primaryKey -o tsv
Write-Host "🔑 Key: in AI_GATEWAY_API_KEY ($($env:AI_GATEWAY_API_KEY.Length) characters, not shown)"az aigateway model list -g "$RG" --gateway-name "$GATEWAY" --query "[].{name:name, provider:properties.providerKind}" -o table
lists your deployment with the provider Foundry, the alias command returned your model's ID as target and
printed 🔖 Alias gpt-5-mini is ready, and
az aigateway api-key list -g "$RG" --gateway-name "$GATEWAY" --query "[].{name:name, state:properties.state}" -o table
shows app-first-call as active, next to the built-in master key.
What you need: the one-time setup from Before you begin. The template assigns a role, so you need Owner on
the subscription, or Contributor and Role Based Access Control Administrator. This track doesn't use your
gateway from lab 01: azd up creates a new one, so it works on its own.
Review the templates
infra/main.bicep creates the resource group rg-<environment name> and composes three templates. It calls lab
01's gateway.bicep for the gateway, infra/foundry.bicep for the
Foundry account and its gpt-5-mini deployment, and then infra/publish-model.bicep to publish the deployment
through the gateway:
// Lab 02 with azd: "azd up" creates the resource group rg-<environment name> with lab 01's gateway and this lab's
// Foundry account and gpt-5-mini deployment, then publishes the deployment through the gateway: the Foundry User
// role, the model provider, the backend model, the alias, and the app-first-call key.
// The runtime API key is never an output.
targetScope = 'subscription'
@description('azd environment name. The resource group is rg-<environmentName>.')
@minLength(1)
@maxLength(60)
param environmentName string
@description('Region for every resource. Pick one where AI Gateway and the lab\'s Foundry models are available. List them with az provider show --namespace Microsoft.ApiManagement (Setup guide, Find the available regions).')
@metadata({ azd: { type: 'location' } })
param location string
@description('Tokens per minute that each caller identity can use on the model. azd reads it from TOKENS_PER_MINUTE.')
@minValue(1)
param tokensPerMinute int = 10000
var suffix = take(uniqueString(subscription().id, environmentName, location), 6)
var tags = { 'azd-env-name': environmentName }
resource rg 'Microsoft.Resources/resourceGroups@2024-03-01' = {
name: 'rg-${environmentName}'
location: location
tags: tags
}
// Lab 01's template: the gateway with a system-assigned identity.
module gateway '../../01-provision-gateway/infra/gateway.bicep' = {
name: 'aigw-gateway'
scope: rg
params: {
gatewayName: 'aigw-${suffix}'
location: location
tags: tags
}
}
// The backend: a Foundry account with a gpt-5-mini deployment.
module foundry 'foundry.bicep' = {
name: 'aigw-foundry'
scope: rg
params: {
accountName: 'aigw${suffix}'
location: location
tags: tags
}
}
// The Foundry User role for the gateway, then the provider, the backend model, the alias, and the key.
module publish 'publish-model.bicep' = {
name: 'aigw-publish-model'
scope: rg
params: {
gatewayName: gateway.outputs.gatewayName
foundryAccountId: foundry.outputs.accountId
deploymentName: foundry.outputs.deploymentName
tokensPerMinute: tokensPerMinute
}
}
output AZURE_RESOURCE_GROUP string = rg.name
output AI_GATEWAY_NAME string = gateway.outputs.gatewayName
output AI_GATEWAY_ID string = gateway.outputs.gatewayId
output AI_GATEWAY_URL string = gateway.outputs.gatewayUrl
output AI_GATEWAY_PROVIDER string = publish.outputs.providerName
output AI_GATEWAY_BACKEND_MODEL string = publish.outputs.modelName
output AI_GATEWAY_MODEL string = publish.outputs.aliasName
output AI_GATEWAY_API_KEY_NAME string = publish.outputs.apiKeyName
output FOUNDRY_ACCOUNT_NAME string = foundry.outputs.accountNameinfra/foundry.bicep creates an Azure AI Services account with local key authentication disabled, and a
gpt-5-mini GlobalStandard deployment. Skip it in your own templates if you already have a deployment:
// Optional: creates a Microsoft Foundry (Azure AI Services) account with one gpt-5-mini deployment
// to use as the backend for this lab. Skip it if you already have a Foundry or Azure OpenAI deployment.
// Mirrors infra/foundry-models/main.bicep in the reference sample. Model inference is billed.
// The lab's main.bicep calls it with azd. You can also deploy it on its own with "az deployment group create".
targetScope = 'resourceGroup'
@description('Foundry account name. It is also used as the custom subdomain, so it must be globally unique.')
@minLength(2)
@maxLength(64)
param accountName string
@description('Region for the Foundry account. The model and SKU must be available in this region.')
param location string = resourceGroup().location
@description('Name of the model deployment that the gateway will publish.')
param deploymentName string = 'gpt-5-mini'
@description('Model to deploy.')
param modelName string = 'gpt-5-mini'
@description('Model version to deploy.')
param modelVersion string = '2025-08-07'
@description('Deployment SKU.')
param skuName string = 'GlobalStandard'
@description('Deployment capacity in thousands of tokens per minute.')
@minValue(1)
param capacity int = 10
@description('Tags applied to the account.')
param tags object = {}
resource account 'Microsoft.CognitiveServices/accounts@2025-04-01-preview' = {
name: accountName
location: location
tags: tags
kind: 'AIServices'
sku: {
name: 'S0'
}
identity: {
type: 'SystemAssigned'
}
properties: {
allowProjectManagement: false
customSubDomainName: toLower(accountName)
disableLocalAuth: true
networkAcls: {
defaultAction: 'Allow'
virtualNetworkRules: []
ipRules: []
}
publicNetworkAccess: 'Enabled'
}
}
resource deployment 'Microsoft.CognitiveServices/accounts/deployments@2025-04-01-preview' = {
parent: account
name: deploymentName
sku: {
name: skuName
capacity: capacity
}
properties: {
model: {
name: modelName
format: 'OpenAI'
version: modelVersion
}
}
}
output accountId string = account.id
output accountName string = account.name
output endpoint string = account.properties.endpoint
output deploymentName string = deployment.name
output deploymentId string = deployment.idinfra/publish-model.bicep adds to the gateway. It grants the gateway's identity the Foundry User role on the
account (in the account's own resource group, with
foundry-user-role.bicep), and only then creates a Foundry model provider that
authenticates with the managed identity. It registers the deployment as a backend model with a per-caller token
limit, publishes an alias, and creates a runtime API key. It only references the gateway and the account, so you
can also deploy it on its own to a gateway you already have, with az deployment group create:
// Publishes one Microsoft Foundry model deployment through an existing AI Gateway:
// grants the gateway's managed identity the Foundry User role on the account, creates a Foundry model
// provider that authenticates with that identity, registers the deployment as a backend model, publishes a
// callable alias for it, and creates a runtime API key for the first app.
// Mirrors the Foundry provider and model registration in infra/ai-gateway/main.bicep of the reference sample.
// The lab's main.bicep calls it with azd, and other labs call it as a module. You can also deploy it on its own
// to a gateway you already have with "az deployment group create".
targetScope = 'resourceGroup'
@description('Name of the existing AI Gateway (Microsoft.ApiManagement/aigateways) in this resource group.')
param gatewayName string
@description('Resource ID of the Foundry (Azure AI Services) or Azure OpenAI account. It can be in another resource group or subscription.')
param foundryAccountId string
@description('Name of the model deployment in that account, for example gpt-5-mini.')
param deploymentName string
@description('Version of the deployed model, for example 2025-08-07. Optional.')
param modelVersion string = ''
@description('Name of the model provider to create in the gateway.')
param providerName string = 'foundry-models'
@description('Callable alias that clients send in the request "model" field.')
param aliasName string = deploymentName
@description('Name of the runtime API key to create for the first app.')
param apiKeyName string = 'app-first-call'
@description('Display name of the runtime API key.')
param apiKeyDisplayName string = 'First call app key'
@description('Tokens per minute that each caller identity can use on this model.')
@minValue(1)
param tokensPerMinute int = 10000
var foundryAccountName = last(split(foundryAccountId, '/'))
var foundrySubscriptionId = split(foundryAccountId, '/')[2]
var foundryResourceGroup = split(foundryAccountId, '/')[4]
var deploymentId = '${foundryAccountId}/deployments/${deploymentName}'
#disable-next-line BCP081
resource aiGateway 'Microsoft.ApiManagement/aigateways@2026-11-01' existing = {
name: gatewayName
}
#disable-next-line BCP081
resource defaultWorkspace 'Microsoft.ApiManagement/aigateways/workspaces@2026-11-01' existing = {
parent: aiGateway
name: 'default'
}
resource foundryAccount 'Microsoft.CognitiveServices/accounts@2025-04-01-preview' existing = {
scope: resourceGroup(foundrySubscriptionId, foundryResourceGroup)
name: foundryAccountName
}
var foundryEndpoint = foundryAccount.properties.endpoint
var normalizedFoundryEndpoint = endsWith(foundryEndpoint, '/') ? foundryEndpoint : '${foundryEndpoint}/'
module foundryUserRole './foundry-user-role.bicep' = {
name: take('foundry-user-${uniqueString(aiGateway.id, foundryAccountId)}', 64)
scope: resourceGroup(foundrySubscriptionId, foundryResourceGroup)
params: {
foundryAccountName: foundryAccountName
gatewayPrincipalId: aiGateway.identity.principalId
gatewayResourceId: aiGateway.id
}
}
#disable-next-line BCP081
resource foundryProvider 'Microsoft.ApiManagement/aigateways/workspaces/modelProviders@2026-11-01' = {
parent: defaultWorkspace
name: providerName
dependsOn: [
foundryUserRole
]
properties: {
modelProviderType: 'Foundry'
displayName: 'Foundry models'
description: 'Managed-identity Foundry provider created by the first-model-call lab.'
foundry: {
endpoint: {
url: normalizedFoundryEndpoint
}
resourceIds: [
foundryAccountId
]
}
authentication: {
managedIdentity: {
resource: 'https://cognitiveservices.azure.com/'
}
}
}
}
#disable-next-line BCP081
resource model 'Microsoft.ApiManagement/aigateways/workspaces/modelProviders/models@2026-11-01' = {
parent: foundryProvider
name: deploymentName
properties: {
displayName: deploymentName
description: 'Foundry deployment ${deploymentName} published through AI Gateway.'
supportedEndpoints: [
'/openai/v1/chat/completions'
'/openai/v1/responses'
]
deployment: {
resourceId: deploymentId
modelName: deploymentName
modelVersion: empty(modelVersion) ? null : modelVersion
}
policies: [
{
type: 'tokenLimit'
period: 'minute'
count: tokensPerMinute
counterKey: [
'identity'
]
}
]
}
}
#disable-next-line BCP081
resource alias 'Microsoft.ApiManagement/aigateways/workspaces/aliases@2026-11-01' = {
parent: defaultWorkspace
name: aliasName
properties: {
backendModels: [
{ resourceId: model.id }
]
}
}
#disable-next-line BCP081
resource runtimeApiKey 'Microsoft.ApiManagement/aigateways/apiKeys@2026-11-01' = {
parent: aiGateway
name: apiKeyName
properties: {
displayName: apiKeyDisplayName
}
}
output providerName string = foundryProvider.name
output modelName string = model.name
output aliasName string = alias.name
output apiKeyName string = runtimeApiKey.name
output gatewayUrl string = 'https://${aiGateway.properties.frontend.defaultHostname}'
output modelEndpoint string = 'https://${aiGateway.properties.frontend.defaultHostname}/default/models/openai/v1'
output foundryUserRoleAssignmentName string = foundryUserRole.outputs.roleAssignmentNameThe templates use the 2026-11-01 management API. The provider sets modelProviderType: 'Foundry', the account's
endpoint in foundry.endpoint.url, and the managed identity in a top-level authentication object. The model lists
the API paths it serves in supportedEndpoints, and points to the deployment with deployment.resourceId,
deployment.modelName, and an optional deployment.modelVersion. The provider and the model both need a
displayName. The CLI track's az aigateway commands call the same API version, so their output shows the same
names, such as modelProviderType and foundry.endpoint.url. Only the --foundry input of
az aigateway model-provider create keeps a shorter shape: the endpoint as a string, and the authentication with a
kind.
Deploy with azd up
Move into the lab's folder, create an azd environment named aigw-lab02 in your current subscription, then provision
everything. It takes about three minutes.
# Move into the lab's folder, which holds the azd project
cd ~/aigateway/labs/02-first-model-call &&
echo "📂 Lab folder: $(pwd)"
# Create an azd environment in your current subscription, so that azd doesn't prompt you
azd env new aigw-lab02 --location eastus2 --subscription "$(az account show --query id --output tsv)" &&
echo "📝 azd environment aigw-lab02 deploys to eastus2"
# Provision the gateway, the Foundry deployment, the role, the provider, the model, the alias, and the key
azd up &&
echo "🚀 Gateway $(azd env get-value AI_GATEWAY_NAME) is deployed"# Move into the lab's folder, which holds the azd project
Set-Location ~/aigateway/labs/02-first-model-call
if ($?) { Write-Host "📂 Lab folder: $(Get-Location)" }
# Create an azd environment in your current subscription, so that azd doesn't prompt you
azd env new aigw-lab02 --location eastus2 --subscription (az account show --query id --output tsv)
if ($?) { Write-Host "📝 azd environment aigw-lab02 deploys to eastus2" }
# Provision the gateway, the Foundry deployment, the role, the provider, the model, the alias, and the key
azd up
if ($?) { Write-Host "🚀 Gateway $(azd env get-value AI_GATEWAY_NAME) is deployed" }For another region, pick one where AI Gateway and the gpt-5-mini model are available, as in Find the available
regions.
Read the outputs and the key
Set the variables from Before you begin, which the next sections and Verify use, from azd's environment values. Template outputs never contain the key value, so read it from the gateway straight into a variable, where it never shows on screen.
# Read the names that the next sections and Verify use from azd's environment values
export RG="$(azd env get-value AZURE_RESOURCE_GROUP)"
export GATEWAY="$(azd env get-value AI_GATEWAY_NAME)"
export GATEWAY_ID="$(azd env get-value AI_GATEWAY_ID)"
export FOUNDRY_RG="$RG"
export FOUNDRY_ACCOUNT="$(azd env get-value FOUNDRY_ACCOUNT_NAME)"
export DEPLOYMENT="$(azd env get-value AI_GATEWAY_BACKEND_MODEL)"
export PROVIDER="$(azd env get-value AI_GATEWAY_PROVIDER)"
echo "📝 Gateway $GATEWAY and Foundry account $FOUNDRY_ACCOUNT, in $RG"
# Read the gateway's runtime URL and the alias
export AI_GATEWAY_URL="$(azd env get-value AI_GATEWAY_URL)"
export AI_GATEWAY_MODEL="$(azd env get-value AI_GATEWAY_MODEL)"
# Read the key from the gateway, so that it never lands in a file or a deployment log
export AI_GATEWAY_API_KEY="$(az aigateway api-key list-secrets --ids "$GATEWAY_ID/apiKeys/app-first-call" \
--query primaryKey --output tsv)"
echo "🌐 Runtime URL: $AI_GATEWAY_URL"
echo "🔖 Model alias: $AI_GATEWAY_MODEL"
echo "🔑 Key: in AI_GATEWAY_API_KEY (${#AI_GATEWAY_API_KEY} characters, not shown)"# Read the names that the next sections and Verify use from azd's environment values
$RG = azd env get-value AZURE_RESOURCE_GROUP
$GATEWAY = azd env get-value AI_GATEWAY_NAME
$GATEWAY_ID = azd env get-value AI_GATEWAY_ID
$FOUNDRY_RG = $RG
$FOUNDRY_ACCOUNT = azd env get-value FOUNDRY_ACCOUNT_NAME
$DEPLOYMENT = azd env get-value AI_GATEWAY_BACKEND_MODEL
$PROVIDER = azd env get-value AI_GATEWAY_PROVIDER
Write-Host "📝 Gateway $GATEWAY and Foundry account $FOUNDRY_ACCOUNT, in $RG"
# Read the gateway's runtime URL and the alias
$env:AI_GATEWAY_URL = azd env get-value AI_GATEWAY_URL
$env:AI_GATEWAY_MODEL = azd env get-value AI_GATEWAY_MODEL
# Read the key from the gateway, so that it never lands in a file or a deployment log
$env:AI_GATEWAY_API_KEY = az aigateway api-key list-secrets --ids "$GATEWAY_ID/apiKeys/app-first-call" `
--query primaryKey --output tsv
Write-Host "🌐 Runtime URL: $env:AI_GATEWAY_URL"
Write-Host "🔖 Model alias: $env:AI_GATEWAY_MODEL"
Write-Host "🔑 Key: in AI_GATEWAY_API_KEY ($($env:AI_GATEWAY_API_KEY.Length) characters, not shown)"azd up ended with SUCCESS, and the last commands printed 🔖 Model alias: gpt-5-mini. In the portal, select
the new gateway in the gateway switcher: Models lists gpt-5-mini.
Change the token limit
The template gives each caller 10,000 tokens per minute on the model. The value comes from the
TOKENS_PER_MINUTE azd environment value, with the default in infra/main.parameters.json. To change it, set the
value and provision again. Only the model changes.
# Set the tokens per minute that each caller can use on the model
azd env set TOKENS_PER_MINUTE 20000 &&
echo "📝 TOKENS_PER_MINUTE is 20000"
# Provision again, which updates only the model
azd provision &&
echo "🔐 Token limit of 20,000 tokens per minute per caller is on $DEPLOYMENT"# Set the tokens per minute that each caller can use on the model
azd env set TOKENS_PER_MINUTE 20000
if ($?) { Write-Host "📝 TOKENS_PER_MINUTE is 20000" }
# Provision again, which updates only the model
azd provision
if ($?) { Write-Host "🔐 Token limit of 20,000 tokens per minute per caller is on $DEPLOYMENT" }What you need: the one-time setup from Before you begin, plus Terraform 1.9 or later, as in
Deploy a lab with Terraform. The configuration assigns a role, so you need
Owner on the subscription, or Contributor and Role Based Access Control Administrator. This track doesn't
use your gateway from lab 01: terraform apply creates a new one, so it works on its own.
Review the configuration
The configuration mirrors the Bicep templates, one file each. terraform/main.tf creates the resource group
rg-<environment_name> and calls lab 01's gateway module for the
gateway:
# Lab 02 with Terraform: creates the resource group rg-<environment_name> with lab 01's gateway and this lab's
# Foundry account and gpt-5-mini deployment (foundry.tf), then publishes the deployment through the gateway: the
# Foundry User role (foundry-user-role.tf), the model provider, the backend model, the alias, and the
# app-first-call key (publish-model.tf). The runtime API key is never an output.
data "azapi_client_config" "current" {}
locals {
suffix = substr(sha1("${data.azapi_client_config.current.subscription_id}/${var.environment_name}/${var.location}"), 0, 6)
tags = { "environment-name" = var.environment_name }
}
resource "azurerm_resource_group" "rg" {
name = "rg-${var.environment_name}"
location = var.location
tags = local.tags
}
# Lab 01's module: the gateway with a system-assigned identity.
module "gateway" {
source = "../../01-provision-gateway/terraform/gateway"
gateway_name = "aigw-${local.suffix}"
resource_group_id = azurerm_resource_group.rg.id
location = var.location
tags = local.tags
}terraform/foundry.tf creates the Foundry account, with local key authentication disabled, and its gpt-5-mini
GlobalStandard deployment, with the azurerm provider:
# The backend: a Microsoft Foundry (Azure AI Services) account with one gpt-5-mini deployment. Local auth is off,
# so only Entra identities, such as the gateway's managed identity, can call it. Model inference is billed.
locals {
model_name = "gpt-5-mini"
model_version = "2025-08-07"
}
resource "azurerm_cognitive_account" "foundry" {
name = "aigw${local.suffix}"
resource_group_name = azurerm_resource_group.rg.name
location = var.location
tags = local.tags
kind = "AIServices"
sku_name = "S0"
custom_subdomain_name = "aigw${local.suffix}"
local_auth_enabled = false
public_network_access_enabled = true
identity {
type = "SystemAssigned"
}
}
resource "azurerm_cognitive_deployment" "model" {
name = local.model_name
cognitive_account_id = azurerm_cognitive_account.foundry.id
sku {
name = "GlobalStandard"
capacity = 10
}
model {
format = "OpenAI"
name = local.model_name
version = local.model_version
}
}terraform/foundry-user-role.tf grants the gateway's identity the Foundry User role on the account:
# Grants the gateway's system-assigned managed identity the Foundry User role on the Foundry account, so the
# gateway can call the account's model deployments without keys.
locals {
foundry_user_role_id = "53ca6127-db72-4b80-b1b0-d745d6d5456d"
}
resource "azurerm_role_assignment" "foundry_user" {
name = uuidv5("url", "${azurerm_cognitive_account.foundry.id}/${module.gateway.gateway_id}/${local.foundry_user_role_id}")
scope = azurerm_cognitive_account.foundry.id
role_definition_id = "/subscriptions/${data.azapi_client_config.current.subscription_id}/providers/Microsoft.Authorization/roleDefinitions/${local.foundry_user_role_id}"
principal_id = module.gateway.principal_id
principal_type = "ServicePrincipal"
skip_service_principal_aad_check = true
}terraform/publish-model.tf publishes the deployment with the azapi provider, which sends the AI Gateway API
version, 2026-11-01, as is. It creates the Foundry model provider only after the role assignment, then the backend
model with its per-caller token limit, the alias, and the key. terraform/variables.tf declares the token limit,
tokens_per_minute, with the same default as the Bicep template:
# Publishes the Foundry deployment through the gateway: a Foundry model provider that signs in with the gateway's
# managed identity, the deployment as a backend model with a per-caller token limit, an alias that apps send in
# the "model" field, and a runtime API key for the first app. Read the key with az aigateway api-key list-secrets.
locals {
workspace_id = "${module.gateway.gateway_id}/workspaces/default"
}
resource "azapi_resource" "provider" {
type = "Microsoft.ApiManagement/aigateways/workspaces/modelProviders@2026-11-01"
name = "foundry-models"
parent_id = local.workspace_id
body = {
properties = {
modelProviderType = "Foundry"
displayName = "Foundry models"
description = "Managed-identity Foundry provider created by the first-model-call lab."
foundry = {
endpoint = {
url = "${trimsuffix(azurerm_cognitive_account.foundry.endpoint, "/")}/"
}
resourceIds = [azurerm_cognitive_account.foundry.id]
}
authentication = {
managedIdentity = {
resource = "https://cognitiveservices.azure.com/"
}
}
}
}
schema_validation_enabled = false
depends_on = [azurerm_role_assignment.foundry_user]
}
resource "azapi_resource" "model" {
type = "Microsoft.ApiManagement/aigateways/workspaces/modelProviders/models@2026-11-01"
name = azurerm_cognitive_deployment.model.name
parent_id = azapi_resource.provider.id
body = {
properties = {
displayName = azurerm_cognitive_deployment.model.name
description = "Foundry deployment ${azurerm_cognitive_deployment.model.name} published through AI Gateway."
supportedEndpoints = [
"/openai/v1/chat/completions",
"/openai/v1/responses",
]
deployment = {
resourceId = azurerm_cognitive_deployment.model.id
modelName = azurerm_cognitive_deployment.model.name
}
policies = [
{
type = "tokenLimit"
period = "minute"
count = var.tokens_per_minute
counterKey = ["identity"]
}
]
}
}
schema_validation_enabled = false
}
resource "azapi_resource" "alias" {
type = "Microsoft.ApiManagement/aigateways/workspaces/aliases@2026-11-01"
name = azapi_resource.model.name
parent_id = local.workspace_id
body = {
properties = {
backendModels = [
{ resourceId = azapi_resource.model.id }
]
}
}
schema_validation_enabled = false
}
resource "azapi_resource" "api_key" {
type = "Microsoft.ApiManagement/aigateways/apiKeys@2026-11-01"
name = "app-first-call"
parent_id = module.gateway.gateway_id
body = {
properties = {
displayName = "First call app key"
}
}
schema_validation_enabled = false
}Deploy with terraform apply
Move into the lab's Terraform folder, select your current subscription, then plan and apply the configuration. It takes about three minutes, most of it while the new gateway activates.
# Move into the lab's Terraform folder
cd ~/aigateway/labs/02-first-model-call/terraform &&
echo "📂 Terraform folder: $(pwd)"
# Deploy to the Azure CLI's current subscription
export ARM_SUBSCRIPTION_ID="$(az account show --query id --output tsv)" &&
echo "📍 Subscription: $ARM_SUBSCRIPTION_ID"
# Download the providers, then plan and apply the gateway, the Foundry deployment, the role, the provider, the
# model, the alias, and the key
terraform init &&
terraform plan -out tfplan &&
terraform apply tfplan &&
echo "🚀 Gateway $(terraform output -raw ai_gateway_name) is deployed"# Move into the lab's Terraform folder
Set-Location ~/aigateway/labs/02-first-model-call/terraform
if ($?) { Write-Host "📂 Terraform folder: $(Get-Location)" }
# Deploy to the Azure CLI's current subscription
$env:ARM_SUBSCRIPTION_ID = az account show --query id --output tsv
if ($?) { Write-Host "📍 Subscription: $env:ARM_SUBSCRIPTION_ID" }
# Download the providers, then plan and apply the gateway, the Foundry deployment, the role, the provider, the
# model, the alias, and the key
terraform init
if ($?) { terraform plan -out tfplan }
if ($?) { terraform apply tfplan }
if ($?) { Write-Host "🚀 Gateway $(terraform output -raw ai_gateway_name) is deployed" }The configuration deploys to eastus2. For another region where AI Gateway and the gpt-5-mini model are
available, as in Find the available regions, run
export TF_VAR_location=<region> (PowerShell: $env:TF_VAR_location = "<region>") before terraform plan.
Read the outputs and the key
Set the variables from Before you begin, which the next sections and Verify use, from the Terraform outputs. The outputs never contain the key value, so read it from the gateway straight into a variable, where it never shows on screen or lands in the Terraform state.
# Read the names that the next sections and Verify use from the Terraform outputs
export RG="$(terraform output -raw azure_resource_group)"
export GATEWAY="$(terraform output -raw ai_gateway_name)"
export GATEWAY_ID="$(terraform output -raw ai_gateway_id)"
export FOUNDRY_RG="$RG"
export FOUNDRY_ACCOUNT="$(terraform output -raw foundry_account_name)"
export DEPLOYMENT="$(terraform output -raw ai_gateway_backend_model)"
export PROVIDER="$(terraform output -raw ai_gateway_provider)"
echo "📝 Gateway $GATEWAY and Foundry account $FOUNDRY_ACCOUNT, in $RG"
# Read the gateway's runtime URL and the alias
export AI_GATEWAY_URL="$(terraform output -raw ai_gateway_url)"
export AI_GATEWAY_MODEL="$(terraform output -raw ai_gateway_model)"
# Read the key from the gateway, so that it never lands in a file or the Terraform state
export AI_GATEWAY_API_KEY="$(az aigateway api-key list-secrets --ids "$GATEWAY_ID/apiKeys/app-first-call" \
--query primaryKey --output tsv)"
echo "🌐 Runtime URL: $AI_GATEWAY_URL"
echo "🔖 Model alias: $AI_GATEWAY_MODEL"
echo "🔑 Key: in AI_GATEWAY_API_KEY (${#AI_GATEWAY_API_KEY} characters, not shown)"
# Move back to the lab's folder, which holds the scripts and samples
cd .. && echo "📂 Lab folder: $(pwd)"# Read the names that the next sections and Verify use from the Terraform outputs
$RG = terraform output -raw azure_resource_group
$GATEWAY = terraform output -raw ai_gateway_name
$GATEWAY_ID = terraform output -raw ai_gateway_id
$FOUNDRY_RG = $RG
$FOUNDRY_ACCOUNT = terraform output -raw foundry_account_name
$DEPLOYMENT = terraform output -raw ai_gateway_backend_model
$PROVIDER = terraform output -raw ai_gateway_provider
Write-Host "📝 Gateway $GATEWAY and Foundry account $FOUNDRY_ACCOUNT, in $RG"
# Read the gateway's runtime URL and the alias
$env:AI_GATEWAY_URL = terraform output -raw ai_gateway_url
$env:AI_GATEWAY_MODEL = terraform output -raw ai_gateway_model
# Read the key from the gateway, so that it never lands in a file or the Terraform state
$env:AI_GATEWAY_API_KEY = az aigateway api-key list-secrets --ids "$GATEWAY_ID/apiKeys/app-first-call" `
--query primaryKey --output tsv
Write-Host "🌐 Runtime URL: $env:AI_GATEWAY_URL"
Write-Host "🔖 Model alias: $env:AI_GATEWAY_MODEL"
Write-Host "🔑 Key: in AI_GATEWAY_API_KEY ($($env:AI_GATEWAY_API_KEY.Length) characters, not shown)"
# Move back to the lab's folder, which holds the scripts and samples
Set-Location ..
if ($?) { Write-Host "📂 Lab folder: $(Get-Location)" }terraform apply ended with Apply complete! Resources: 9 added, 0 changed, 0 destroyed., and the output
shows 🔖 Model alias: gpt-5-mini. In the portal, select the new gateway in the gateway switcher: Models
lists gpt-5-mini.
The next sections run from the lab's folder, with -chdir=terraform for the terraform commands.
Change the token limit
The configuration gives each caller 10,000 tokens per minute on the model. The value is the tokens_per_minute
variable, with the default in terraform/variables.tf. To change it, pass it to terraform plan and apply the
plan. Only the model changes.
# Plan the new token limit: 20,000 tokens per minute for each caller
terraform -chdir=terraform plan -var tokens_per_minute=20000 -out tfplan &&
echo "📝 The plan updates only the model"
# Apply the plan
terraform -chdir=terraform apply tfplan &&
echo "🔐 Token limit of 20,000 tokens per minute per caller is on $DEPLOYMENT"# Plan the new token limit: 20,000 tokens per minute for each caller
terraform -chdir=terraform plan -var tokens_per_minute=20000 -out tfplan
if ($?) { Write-Host "📝 The plan updates only the model" }
# Apply the plan
terraform -chdir=terraform apply tfplan
if ($?) { Write-Host "🔐 Token limit of 20,000 tokens per minute per caller is on $DEPLOYMENT" }The plan ended with Plan: 0 to add, 1 to change, 0 to destroy., for azapi_resource.model.
A later terraform plan or terraform apply without -var goes back to the default, 10,000.
Role assignments can take a few minutes to take effect. Until then, calls can fail with 403 from the
backend. Wait and retry.
Make your first call
Use the same alias and key everywhere. The gateway exposes the OpenAI-compatible base URL
<gateway URL>/default/models/openai/v1, so OpenAI SDKs only need a different base URL and the runtime key.
Try the model in Catalog
- Select Catalog and pick your model.
- Under Authentication, check that Method is API key, and choose your key as the API key, for
example
app-first-call. - Send a prompt in the built-in playground, for example
Write a one-sentence status update. - Select Continue in code for the endpoint URL and snippets for your app: cURL, the OpenAI SDK, LangChain, Vercel AI, the GitHub Copilot SDK, and Microsoft Agent Framework.
If the playground asks you to Sign in again, your portal session expired. Sign in and resend the prompt.
Call Chat Completions
# Call Chat Completions through the gateway: the alias in model, and the runtime key as a bearer token
curl -sS "${AI_GATEWAY_URL%/}/default/models/openai/v1/chat/completions" \
-H "Authorization: Bearer $AI_GATEWAY_API_KEY" -H "Content-Type: application/json" \
-d "{\"model\":\"$AI_GATEWAY_MODEL\",\"messages\":[{\"role\":\"user\",\"content\":\"Write a one-sentence status update.\"}]}" \
| jq -r 'if .choices then "🤖 \(.choices[0].message.content)" else "🚨 \(.error.message // .)" end'# Build the request: the alias in model, and one user message
$body = @{
model = $env:AI_GATEWAY_MODEL
messages = @(@{ role = "user"; content = "Write a one-sentence status update." })
} | ConvertTo-Json -Depth 5
# Call Chat Completions through the gateway, with the runtime key as a bearer token
$reply = Invoke-RestMethod -Method Post `
-Uri "$($env:AI_GATEWAY_URL.TrimEnd('/'))/default/models/openai/v1/chat/completions" `
-Headers @{ Authorization = "Bearer $env:AI_GATEWAY_API_KEY" } -ContentType "application/json" -Body $body
if ($?) { Write-Host "🤖 $($reply.choices[0].message.content)" }Call the Responses API
# Call the Responses API through the gateway, with the same alias and key, and print the reply's text
curl -sS "${AI_GATEWAY_URL%/}/default/models/openai/v1/responses" \
-H "Authorization: Bearer $AI_GATEWAY_API_KEY" -H "Content-Type: application/json" \
-d "{\"model\":\"$AI_GATEWAY_MODEL\",\"input\":\"Summarize what an AI gateway does in one sentence.\"}" \
| jq -r 'if .output then "🤖 \([.output[] | select(.type == "message") | .content[] | select(.type == "output_text") | .text] | join(""))" else "🚨 \(.error.message // .)" end'# Build the request: the alias in model, and the prompt in input
$body = @{ model = $env:AI_GATEWAY_MODEL; input = "Summarize what an AI gateway does in one sentence." } |
ConvertTo-Json
# Call the Responses API through the gateway, and print the text of the reply's message
$reply = Invoke-RestMethod -Method Post `
-Uri "$($env:AI_GATEWAY_URL.TrimEnd('/'))/default/models/openai/v1/responses" `
-Headers @{ Authorization = "Bearer $env:AI_GATEWAY_API_KEY" } -ContentType "application/json" -Body $body
if ($?) { Write-Host "🤖 $(($reply.output | Where-Object type -eq "message").content.text -join '')" }Run both calls with a script
The script prints the HTTP status and the reply for each interface, which makes failures easier to read.
#!/usr/bin/env bash
# Calls a model through AI Gateway twice, once with Chat Completions and once with the Responses API,
# and prints the HTTP status and the model's reply for each call.
#
# Usage:
# export AI_GATEWAY_URL="<gateway-url>"
# export AI_GATEWAY_MODEL="<model-name>"
# export AI_GATEWAY_API_KEY="..." # never commit this value
# ./call-model.sh ["Your prompt"]
# Requires curl and jq.
set -euo pipefail
PROMPT="${1:-Write a one-sentence status update.}"
: "${AI_GATEWAY_URL:?Set AI_GATEWAY_URL to the gateway runtime URL, for example https://aigw-labs-k3v7qz2m9xw4p.eastus2-01.ai.gateway.azure.com}"
: "${AI_GATEWAY_MODEL:?Set AI_GATEWAY_MODEL to the callable model alias}"
: "${AI_GATEWAY_API_KEY:?Set AI_GATEWAY_API_KEY to a runtime key from Access > API keys}"
BASE="${AI_GATEWAY_URL%/}/default/models/openai/v1"
call() {
local label="$1" path="$2" body="$3" filter="$4" out status reply
out=$(curl -sS -w '\n%{http_code}' "$BASE$path" \
-H "Authorization: Bearer $AI_GATEWAY_API_KEY" -H "Content-Type: application/json" -d "$body")
status="${out##*$'\n'}"
reply="${out%$'\n'*}"
if [[ "$status" == 2* ]]; then
printf '%-17s HTTP %s %s\n' "$label" "$status" "$(printf '%s' "$reply" | jq -r "$filter")"
else
printf '%-17s HTTP %s %s\n' "$label" "$status" "$reply"
fi
}
call "Chat Completions" "/chat/completions" \
"$(jq -n --arg m "$AI_GATEWAY_MODEL" --arg p "$PROMPT" '{model:$m, messages:[{role:"user", content:$p}]}')" \
'.choices[0].message.content'
call "Responses API" "/responses" \
"$(jq -n --arg m "$AI_GATEWAY_MODEL" --arg p "$PROMPT" '{model:$m, input:$p}')" \
'[.output[]? | select(.type == "message") | .content[]? | select(.type == "output_text") | .text] | join("")'On GitHub, open infra/call-model.sh or infra/call-model.ps1.
# Run the script, which calls both APIs and prints the HTTP status and the reply of each
echo "🤖 Calling $AI_GATEWAY_MODEL with Chat Completions and the Responses API:"
./infra/call-model.sh# Run the script, which calls both APIs and prints the HTTP status and the reply of each
Write-Host "🤖 Calling $env:AI_GATEWAY_MODEL with Chat Completions and the Responses API:"
./infra/call-model.ps1Install the SDK
Each sample reads AI_GATEWAY_URL, AI_GATEWAY_MODEL, and AI_GATEWAY_API_KEY from the environment, so the
key never lives in code. Run the commands from the lab folder.
# Move into the samples folder, and install the OpenAI SDK for Python that requirements.txt lists
cd infra
pip install -r requirements.txt # the openai package
echo "📦 The OpenAI SDK for Python is installed"# Move into the samples folder, and install the OpenAI SDK for TypeScript that package.json lists
cd infra
npm install # the openai npm package
echo "📦 The OpenAI SDK for TypeScript is installed"# Move into the .NET sample, and restore it: it uses only HttpClient and System.Text.Json
cd infra/dotnet
dotnet restore # no extra packages: HttpClient and System.Text.Json
echo "📦 The .NET sample is restored"Call the model from code
The OpenAI SDKs send their API key as Authorization: Bearer <key>, which is where the gateway reads the runtime
key, so the key goes straight into the SDK. The .NET sample uses HttpClient, so it sets the same header itself.
The gateway also accepts the key in an api-key header, which Azure OpenAI clients send, but not both headers in
one request: that returns 400 Specify either an API key or an Authorization header, not both.
import os
from openai import OpenAI
gateway_url = os.environ["AI_GATEWAY_URL"].rstrip("/")
model = os.environ["AI_GATEWAY_MODEL"]
client = OpenAI(
base_url=f"{gateway_url}/default/models/openai/v1",
api_key=os.environ["AI_GATEWAY_API_KEY"],
)
response = client.responses.create(model=model, input="Write a one-sentence status update.")
print(f"Responses API: {response.output_text}")
completion = client.chat.completions.create(
model=model,
messages=[{"role": "user", "content": "Describe an AI gateway in one sentence."}],
)
print(f"Chat Completions: {completion.choices[0].message.content}")import OpenAI from "openai";
const gatewayUrl = process.env.AI_GATEWAY_URL!.replace(/\/+$/, "");
const model = process.env.AI_GATEWAY_MODEL!;
const client = new OpenAI({
baseURL: `${gatewayUrl}/default/models/openai/v1`,
apiKey: process.env.AI_GATEWAY_API_KEY!,
});
const response = await client.responses.create({ model, input: "Write a one-sentence status update." });
console.log(`Responses API: ${response.output_text}`);
const completion = await client.chat.completions.create({
model,
messages: [{ role: "user", content: "Describe an AI gateway in one sentence." }],
});
console.log(`Chat Completions: ${completion.choices[0].message.content}`);using System.Net.Http.Headers;
using System.Net.Http.Json;
using System.Text.Json;
var gatewayUrl = Environment.GetEnvironmentVariable("AI_GATEWAY_URL")!.TrimEnd('/');
var model = Environment.GetEnvironmentVariable("AI_GATEWAY_MODEL")!;
using var http = new HttpClient { BaseAddress = new Uri($"{gatewayUrl}/default/models/openai/v1/") };
http.DefaultRequestHeaders.Authorization =
new AuthenticationHeaderValue("Bearer", Environment.GetEnvironmentVariable("AI_GATEWAY_API_KEY"));
using var reply = await http.PostAsJsonAsync("chat/completions", new
{
model,
messages = new[] { new { role = "user", content = "Describe an AI gateway in one sentence." } },
});
reply.EnsureSuccessStatusCode();
using var json = JsonDocument.Parse(await reply.Content.ReadAsStringAsync());
Console.WriteLine(json.RootElement.GetProperty("choices")[0].GetProperty("message").GetProperty("content").GetString());The full samples in infra also call the Responses API from C# and print token usage: open
first-call.py, first-call.ts, and
dotnet/Program.cs.
Run the sample
# Run the sample, which calls the model through the gateway with the Responses API and Chat Completions
echo "🤖 Replies from the model, through the gateway:"
python first-call.py# Run the sample, which calls the model through the gateway with the Responses API and Chat Completions
echo "🤖 Replies from the model, through the gateway:"
node first-call.ts # Node.js 22.18 or later; on older versions run: npx tsx first-call.ts# Build and run the sample, which calls the model through the gateway with the Responses API and Chat Completions
echo "🤖 Replies from the model, through the gateway:"
dotnet runExpected output (your text differs):
🤖 Replies from the model, through the gateway:
Responses API: All systems are running normally and the release is on track for Friday.
Chat Completions: An AI gateway is a governed entry point that routes, secures, and observes calls to AI models and tools.The model answers through the gateway, and you never used a Foundry key or endpoint.
Verify
Confirm the model is published
# List the models that the gateway publishes, with the kind of provider of each
echo "✨ Models on gateway $GATEWAY:"
az aigateway model list -g "$RG" --gateway-name "$GATEWAY" --query "[].{name:name, provider:properties.providerKind}" -o table# List the models that the gateway publishes, with the kind of provider of each
Write-Host "✨ Models on gateway ${GATEWAY}:"
az aigateway model list -g $RG --gateway-name $GATEWAY --query "[].{name:name, provider:properties.providerKind}" -o tableThe list includes your deployment. In the portal, Catalog lists the model too.
Get a 200 with the key
# Call Chat Completions with the key, and print only the HTTP status
STATUS="$(curl -sS -o /dev/null -w "%{http_code}" "${AI_GATEWAY_URL%/}/default/models/openai/v1/chat/completions" \
-H "Authorization: Bearer $AI_GATEWAY_API_KEY" -H "Content-Type: application/json" \
-d "{\"model\":\"$AI_GATEWAY_MODEL\",\"messages\":[{\"role\":\"user\",\"content\":\"ping\"}]}")" &&
echo "🔑 With the key: HTTP $STATUS"# Build a short request: the alias in model, and one user message
$body = @{ model = $env:AI_GATEWAY_MODEL; messages = @(@{ role = "user"; content = "ping" }) } | ConvertTo-Json -Depth 5
# Call Chat Completions with the key, and print only the HTTP status
$status = (Invoke-WebRequest -Method Post -Uri "$($env:AI_GATEWAY_URL.TrimEnd('/'))/default/models/openai/v1/chat/completions" `
-Headers @{ Authorization = "Bearer $env:AI_GATEWAY_API_KEY" } -ContentType "application/json" -Body $body -SkipHttpErrorCheck).StatusCode
if ($?) { Write-Host "🔑 With the key: HTTP $status" }Expect 🔑 With the key: HTTP 200.
Get a 401 without the key
Send the same request without the Authorization header. The gateway rejects it before any policy runs or the
backend is called.
# Send the same request without the Authorization header, and print only the HTTP status
STATUS="$(curl -sS -o /dev/null -w "%{http_code}" "${AI_GATEWAY_URL%/}/default/models/openai/v1/chat/completions" \
-H "Content-Type: application/json" \
-d "{\"model\":\"$AI_GATEWAY_MODEL\",\"messages\":[{\"role\":\"user\",\"content\":\"ping\"}]}")" &&
echo "🚦 Without a key: HTTP $STATUS"# Send the same request without the Authorization header, and print only the HTTP status
$status = (Invoke-WebRequest -Method Post -Uri "$($env:AI_GATEWAY_URL.TrimEnd('/'))/default/models/openai/v1/chat/completions" `
-ContentType "application/json" -Body $body -SkipHttpErrorCheck).StatusCode
if ($?) { Write-Host "🚦 Without a key: HTTP $status" }Expect 🚦 Without a key: HTTP 401.
Check that the alias is what clients call
Change one character in model and resend the request with the key. The call fails with 404 and
model_not_found because model must match an alias exactly. Backend model names and Foundry deployment names
aren't callable unless an alias with that name exists.
You got 200 with the key, 401 without it, and an error for a name that isn't an alias.
Go further
- Publish more deployments. After you add deployments in Foundry, rerun the portal import, or repeat the CLI track's Register the deployment and Publish an alias steps for each new deployment.
- Add a per-caller token limit. The Bicep + azd track already sets
tokensPerMinute, and its Change the token limit step changes it. Learn how limits behave in Throttle tokens per caller with token limits. - Publish models from other providers. Put OpenAI, Anthropic, Amazon Bedrock, or Google Vertex AI models behind the same gateway in Publish models from any provider behind stable aliases.
Clean up
The portal and CLI tracks of the next labs reuse the gateway, model, alias, and key, so keep them until you finish the labs you plan to do.
Delete what you published
When you're done with the labs, delete the key, the alias, and the provider, then remove the role assignment that the gateway received on the Foundry account. Delete the alias first: deleting the provider also deletes its models, but not the aliases that point at them. Deleting a model from the gateway doesn't delete the Foundry deployment. The commands use the names from the CLI track. If you imported in the portal, use the provider and alias names shown on Models.
# Delete the runtime API key, so that it stops working
az aigateway api-key delete -g "$RG" --gateway-name "$GATEWAY" -n app-first-call --yes &&
echo "🚮 API key app-first-call is deleted"
# Delete the alias first: deleting the provider doesn't delete the aliases that point at its models
az aigateway model-alias delete -g "$RG" --gateway-name "$GATEWAY" -n "$DEPLOYMENT" --yes &&
echo "🚮 Alias $DEPLOYMENT is deleted"
# Delete the provider, with its models. The Foundry deployment stays.
az aigateway model-provider delete -g "$RG" --gateway-name "$GATEWAY" -n "$PROVIDER" --yes &&
echo "🚮 Model provider $PROVIDER is deleted"
# Read the principal ID of the gateway's identity, and the Foundry account's resource ID
PRINCIPAL_ID="$(az aigateway show -g "$RG" -n "$GATEWAY" --query identity.principalId -o tsv)"
FOUNDRY_ID="$(az cognitiveservices account show -g "$FOUNDRY_RG" -n "$FOUNDRY_ACCOUNT" --query id -o tsv)"
echo "👤 Gateway identity: $PRINCIPAL_ID"
# Remove the Foundry User role that the gateway's identity has on the account
az role assignment delete --assignee "$PRINCIPAL_ID" --role "53ca6127-db72-4b80-b1b0-d745d6d5456d" --scope "$FOUNDRY_ID" &&
echo "🚮 The gateway's Foundry User role on $FOUNDRY_ACCOUNT is removed"# Delete the runtime API key, so that it stops working
az aigateway api-key delete -g $RG --gateway-name $GATEWAY -n app-first-call --yes
if ($?) { Write-Host "🚮 API key app-first-call is deleted" }
# Delete the alias first: deleting the provider doesn't delete the aliases that point at its models
az aigateway model-alias delete -g $RG --gateway-name $GATEWAY -n $DEPLOYMENT --yes
if ($?) { Write-Host "🚮 Alias $DEPLOYMENT is deleted" }
# Delete the provider, with its models. The Foundry deployment stays.
az aigateway model-provider delete -g $RG --gateway-name $GATEWAY -n $PROVIDER --yes
if ($?) { Write-Host "🚮 Model provider $PROVIDER is deleted" }
# Read the principal ID of the gateway's identity, and the Foundry account's resource ID
$PRINCIPAL_ID = az aigateway show -g $RG -n $GATEWAY --query identity.principalId -o tsv
$FOUNDRY_ID = az cognitiveservices account show -g $FOUNDRY_RG -n $FOUNDRY_ACCOUNT --query id -o tsv
Write-Host "👤 Gateway identity: $PRINCIPAL_ID"
# Remove the Foundry User role that the gateway's identity has on the account
az role assignment delete --assignee $PRINCIPAL_ID --role "53ca6127-db72-4b80-b1b0-d745d6d5456d" --scope $FOUNDRY_ID
if ($?) { Write-Host "🚮 The gateway's Foundry User role on $FOUNDRY_ACCOUNT is removed" }If you created the Foundry account with infra/foundry.bicep, delete its resource group with
az group delete --name "$FOUNDRY_RG".
Delete everything azd created
You don't need to keep this gateway for the Bicep + azd tracks of later labs: each of them creates its own. If you
continue with the portal or CLI track of lab 03, you can keep it instead:
it has what that lab needs, the gpt-5-mini alias and the app-first-call key, in the resource group
rg-aigw-lab02.
Otherwise, in the aigateway/labs/02-first-model-call folder, delete the resource group with the gateway and the
Foundry account. --purge also purges the soft-deleted Foundry account. It
takes about three minutes.
# Delete everything that azd up created, purge the Foundry account, and don't ask you to confirm
azd down --purge --force &&
echo "🚮 Everything that azd up created is deleted"azd down doesn't touch the gateway from lab 01, because this track created its own.
Delete everything Terraform created
You don't need to keep this gateway for the Terraform tracks of later labs: each of them creates its own. If you
continue with the portal or CLI track of lab 03, you can keep it instead:
it has what that lab needs, the gpt-5-mini alias and the app-first-call key, in the resource group
rg-aigw-lab02-tf.
Otherwise, in the aigateway/labs/02-first-model-call folder, delete the resource group with the gateway and the
Foundry account. The azurerm provider also purges the soft-deleted
Foundry account. It takes about two minutes.
# Delete everything that terraform apply created, and purge the soft-deleted Foundry account
terraform -chdir=terraform destroy -auto-approve &&
echo "🚮 Everything that terraform apply created is deleted"terraform destroy doesn't touch the gateway from lab 01, because this track created its own.
Next steps
- Publish models from any provider behind stable aliases: add other providers and retarget aliases.
- Issue and rotate runtime API keys: give each app its own key and rotate it safely.
- Throttle tokens per caller with token limits: protect the deployment from noisy neighbors.
- Docs: Connect agents to AI Gateway, Authenticate with an API key, and Microsoft Foundry and AI Gateway.