Azure AI Gateway Labs
Documentation Open portal
Foundations · Lab 02

Publish a Foundry model and make your first call

Publish a Microsoft Foundry model deployment through your gateway with its managed identity, create a runtime API key, try the model in the Catalog playground, and call it from curl, PowerShell, Python, TypeScript, and C#.

Beginner 25 min PortalCLIBicepTerraformCodeNotebook Updated
Notebook Source Edit

What you'll do

  • Publish a Foundry model deployment through the gateway, authenticated with the gateway's managed identity
  • Create a runtime API key and keep its value out of code
  • Call the model through the gateway with the Chat Completions and Responses APIs
  • Explain what the gateway does with a request between your app and the model

Before you start

Tracks: Portal, Azure CLI, Bicep + azd, Terraform, Code, Notebook

The first product team wants to use gpt-5-mini, which already runs in your Microsoft Foundry account. You don't want to hand every team the Foundry endpoint and a provider key. In this lab you publish the deployment through your gateway, give the team a runtime API key of its own, and make the first call from the playground, a shell, and code. The gateway authenticates to Foundry with its own managed identity, so the app never sees a Foundry credential.

Before you begin

One-time setupThe Setup guide tasks that this lab uses. Mark each one done once, and the labs stop listing it.

Tip

On the Bicep + azd and Terraform tracks, azd up or terraform apply creates its own gateway, Foundry account, model, and key, so you can skip this section. You need only the one-time setup above, plus the language runtime for the code track if you follow it.

  • Complete Provision your AI Gateway. You need the gateway, its resource group, and its runtime URL.
  • A model deployment in a Microsoft Foundry (Azure AI Services) or Azure OpenAI account, for example gpt-5-mini. If you don't have one, deploy the optional template described below.
  • Permission to grant the gateway access to the account: Owner, User Access Administrator, or Role Based Access Control Administrator on the Foundry account. You also need to change the gateway, for example Contributor on its resource group.
  • For the code track: Python 3.9 or later, Node.js 22.18 or later, or the .NET 8 SDK.
  • Run the scripts and samples from the lab's folder, ~/aigateway/labs/02-first-model-call.

Set these variables once. Every track and the Verify section reuse them.

# Name your gateway and its resource group, the Foundry account and its deployment, and the provider to create
export RG="<resource-group>"
export GATEWAY="<gateway>"
export FOUNDRY_RG="<foundry-resource-group>"      # resource group of the Foundry account
export FOUNDRY_ACCOUNT="<foundry-account-name>"   # the account name, not a project or deployment name
export DEPLOYMENT="gpt-5-mini"                    # the model deployment to publish
export PROVIDER="foundry-models"                  # CLI track: the gateway provider to create
echo "📝 Gateway $GATEWAY in $RG, deployment $DEPLOYMENT in Foundry account $FOUNDRY_ACCOUNT"

# Read the gateway's resource ID, and build its runtime URL from its host name
export GATEWAY_ID="$(az aigateway show -g "$RG" -n "$GATEWAY" --query id -o tsv)"
export AI_GATEWAY_URL="https://$(az aigateway show -g "$RG" -n "$GATEWAY" --query properties.frontend.defaultHostname -o tsv)"
echo "🆔 Gateway ID: $GATEWAY_ID"
echo "🌐 Runtime URL: $AI_GATEWAY_URL"
No Foundry deployment yet? Create one with Bicep.

infra/foundry.bicep creates an Azure AI Services account with a gpt-5-mini GlobalStandard deployment, mirroring the reference sample. Pick a region where the model is available. Model inference is billed per use.

Bash
# Create the resource group for the Foundry account
az group create --name "$FOUNDRY_RG" --location eastus2 --output none &&
  echo "📦 Resource group $FOUNDRY_RG is ready in eastus2"

# Deploy the template: the Foundry account and its gpt-5-mini deployment, then read the deployment's resource ID
DEPLOYMENT_ID="$(az deployment group create -g "$FOUNDRY_RG" \
  --template-file infra/foundry.bicep \
  --parameters accountName="$FOUNDRY_ACCOUNT" deploymentName="$DEPLOYMENT" \
  --query properties.outputs.deploymentId.value -o tsv)" &&
  echo "✨ Model deployment $DEPLOYMENT is ready: $DEPLOYMENT_ID"

How it works

Publishing a model has three parts. A model provider tells the gateway where the Foundry account is and how to authenticate to it. A backend model registers one deployment under that provider. A callable alias is the stable name that clients send in the request model field. Importing from Foundry creates all three for you.

Clients call one runtime URL with a gateway API key. The gateway validates the key before anything else, runs the policies that apply, swaps the caller's key for its own managed identity token, and forwards the request unchanged to Foundry. The response comes back as is, and the gateway records token usage for monitoring.

StepWhat happensWhere you set it
Inbound authenticationAn active runtime key as Authorization: Bearer <key>, which is what the OpenAI SDKs send, or in an api-key header. Without a valid key the call fails with 401, and with both headers it fails with 400AccessAPI keys
Routing/default/models/openai/v1/chat/completions or /responses, and model must equal an aliasModel details and its Aliases tab
PoliciesToken limits, cost limits, and content safety run before the backend callPolicies (labs 06–09)
Backend authenticationThe gateway's managed identity, with the Foundry User role on the accountModel provider
TelemetryTokens, latency, and the API key that made the callMonitoring (lab 13)

Publish the model and create a key

Pick one track. Each track publishes the deployment, gives you a callable alias, and creates a runtime API key. The Bicep + azd and Terraform tracks also create their own gateway and Foundry deployment, so they don't need lab 01's gateway.

Choose your track
1

Import the deployment from Foundry

  1. Open the AI Gateway portal and select your gateway in the top bar.
  2. Select ModelsAddModels from Microsoft Foundry. (On Home, Add from Microsoft Foundry in the Set up your gateway guide starts the same import.)
  3. In Subscriptions and Resource groups, select the subscription and resource group of your Foundry account. The page discovers the deployments automatically. Deployments that are already in this gateway can't be selected again.
  4. Select the gpt-5-mini deployment and keep Managed identity as the Authentication method.
  5. Under Aliases, keep Create an alias for each model selected, then select Add models. When the banner Models and aliases imported shows, select Close.
Note

The gateway calls Foundry with its system-assigned managed identity, which needs the Foundry User role on the account. If your first call returns 403 from the backend, assign the role yourself and wait a few minutes for it to take effect:

Bash
# Read the principal ID of the gateway's managed identity, and the Foundry account's resource ID
PRINCIPAL_ID="$(az aigateway show -g "$RG" -n "$GATEWAY" --query identity.principalId -o tsv)"
FOUNDRY_ID="$(az cognitiveservices account show -g "$FOUNDRY_RG" -n "$FOUNDRY_ACCOUNT" --query id -o tsv)"
echo "👤 Gateway identity: $PRINCIPAL_ID"

# Give the gateway's identity the Foundry User role on the account, so that it can call the model
az role assignment create --assignee-object-id "$PRINCIPAL_ID" --assignee-principal-type ServicePrincipal \
  --role "53ca6127-db72-4b80-b1b0-d745d6d5456d" --scope "$FOUNDRY_ID" --output none &&
  echo "🔐 The gateway's identity has the Foundry User role on $FOUNDRY_ACCOUNT"
2

Copy the callable alias

On Models, the Backend models tab lists gpt-5-mini with its alias. Select the model: its Overview lists the Endpoints that it supports, such as Chat completions and Responses, and its Aliases tab lists the aliases that route to it. Copy the alias exactly as shown, then save it in your shell. Portal imports name it after the provider and the deployment, for example <foundry-account-name>/gpt-5-mini.

# Save the alias that apps send in the model field
export AI_GATEWAY_MODEL="<model-name>"    # the alias from the Aliases tab
echo "🔖 Model alias: $AI_GATEWAY_MODEL"
3

Create a runtime API key

  1. Select Access, then the API keys tab, then Create API key.
  2. In Key name, enter a name for the app that will use it, for example app-first-call, and select Create.
  3. In API key created, select Copy & done to copy the key. You can view, rotate, or delete it later in AccessAPI keys.
  4. Paste it into your shell without echoing it or writing it to a file:
# Paste the key without showing it, and export it for the calls and samples that follow
printf "AI Gateway API key: " && read -rs AI_GATEWAY_API_KEY && export AI_GATEWAY_API_KEY && echo
echo "🔑 Key: in AI_GATEWAY_API_KEY (${#AI_GATEWAY_API_KEY} characters, not shown)"
Checkpoint

Models lists the Foundry deployment, its Aliases tab shows an alias, and AccessAPI keys shows app-first-call.

Important

Role assignments can take a few minutes to take effect. Until then, calls can fail with 403 from the backend. Wait and retry.

Make your first call

Use the same alias and key everywhere. The gateway exposes the OpenAI-compatible base URL <gateway URL>/default/models/openai/v1, so OpenAI SDKs only need a different base URL and the runtime key.

Choose your track
1

Try the model in Catalog

  1. Select Catalog and pick your model.
  2. Under Authentication, check that Method is API key, and choose your key as the API key, for example app-first-call.
  3. Send a prompt in the built-in playground, for example Write a one-sentence status update.
  4. Select Continue in code for the endpoint URL and snippets for your app: cURL, the OpenAI SDK, LangChain, Vercel AI, the GitHub Copilot SDK, and Microsoft Agent Framework.
Tip

If the playground asks you to Sign in again, your portal session expired. Sign in and resend the prompt.

Checkpoint

The model answers through the gateway, and you never used a Foundry key or endpoint.

Verify

1

Confirm the model is published

# List the models that the gateway publishes, with the kind of provider of each
echo "✨ Models on gateway $GATEWAY:"
az aigateway model list -g "$RG" --gateway-name "$GATEWAY" --query "[].{name:name, provider:properties.providerKind}" -o table

The list includes your deployment. In the portal, Catalog lists the model too.

2

Get a 200 with the key

# Call Chat Completions with the key, and print only the HTTP status
STATUS="$(curl -sS -o /dev/null -w "%{http_code}" "${AI_GATEWAY_URL%/}/default/models/openai/v1/chat/completions" \
  -H "Authorization: Bearer $AI_GATEWAY_API_KEY" -H "Content-Type: application/json" \
  -d "{\"model\":\"$AI_GATEWAY_MODEL\",\"messages\":[{\"role\":\"user\",\"content\":\"ping\"}]}")" &&
  echo "🔑 With the key: HTTP $STATUS"

Expect 🔑 With the key: HTTP 200.

3

Get a 401 without the key

Send the same request without the Authorization header. The gateway rejects it before any policy runs or the backend is called.

# Send the same request without the Authorization header, and print only the HTTP status
STATUS="$(curl -sS -o /dev/null -w "%{http_code}" "${AI_GATEWAY_URL%/}/default/models/openai/v1/chat/completions" \
  -H "Content-Type: application/json" \
  -d "{\"model\":\"$AI_GATEWAY_MODEL\",\"messages\":[{\"role\":\"user\",\"content\":\"ping\"}]}")" &&
  echo "🚦 Without a key: HTTP $STATUS"

Expect 🚦 Without a key: HTTP 401.

4

Check that the alias is what clients call

Change one character in model and resend the request with the key. The call fails with 404 and model_not_found because model must match an alias exactly. Backend model names and Foundry deployment names aren't callable unless an alias with that name exists.

Checkpoint

You got 200 with the key, 401 without it, and an error for a name that isn't an alias.

Go further

  • Publish more deployments. After you add deployments in Foundry, rerun the portal import, or repeat the CLI track's Register the deployment and Publish an alias steps for each new deployment.
  • Add a per-caller token limit. The Bicep + azd track already sets tokensPerMinute, and its Change the token limit step changes it. Learn how limits behave in Throttle tokens per caller with token limits.
  • Publish models from other providers. Put OpenAI, Anthropic, Amazon Bedrock, or Google Vertex AI models behind the same gateway in Publish models from any provider behind stable aliases.

Clean up

The portal and CLI tracks of the next labs reuse the gateway, model, alias, and key, so keep them until you finish the labs you plan to do.

Choose your track
1

Delete what you published

When you're done with the labs, delete the key, the alias, and the provider, then remove the role assignment that the gateway received on the Foundry account. Delete the alias first: deleting the provider also deletes its models, but not the aliases that point at them. Deleting a model from the gateway doesn't delete the Foundry deployment. The commands use the names from the CLI track. If you imported in the portal, use the provider and alias names shown on Models.

# Delete the runtime API key, so that it stops working
az aigateway api-key delete -g "$RG" --gateway-name "$GATEWAY" -n app-first-call --yes &&
  echo "🚮 API key app-first-call is deleted"

# Delete the alias first: deleting the provider doesn't delete the aliases that point at its models
az aigateway model-alias delete -g "$RG" --gateway-name "$GATEWAY" -n "$DEPLOYMENT" --yes &&
  echo "🚮 Alias $DEPLOYMENT is deleted"

# Delete the provider, with its models. The Foundry deployment stays.
az aigateway model-provider delete -g "$RG" --gateway-name "$GATEWAY" -n "$PROVIDER" --yes &&
  echo "🚮 Model provider $PROVIDER is deleted"

# Read the principal ID of the gateway's identity, and the Foundry account's resource ID
PRINCIPAL_ID="$(az aigateway show -g "$RG" -n "$GATEWAY" --query identity.principalId -o tsv)"
FOUNDRY_ID="$(az cognitiveservices account show -g "$FOUNDRY_RG" -n "$FOUNDRY_ACCOUNT" --query id -o tsv)"
echo "👤 Gateway identity: $PRINCIPAL_ID"

# Remove the Foundry User role that the gateway's identity has on the account
az role assignment delete --assignee "$PRINCIPAL_ID" --role "53ca6127-db72-4b80-b1b0-d745d6d5456d" --scope "$FOUNDRY_ID" &&
  echo "🚮 The gateway's Foundry User role on $FOUNDRY_ACCOUNT is removed"

If you created the Foundry account with infra/foundry.bicep, delete its resource group with az group delete --name "$FOUNDRY_RG".

Next steps

Finished the lab?

Mark it complete to track your progress across paths. It's saved only in this browser.