Azure AI Gateway Labs
Documentation Open portal
Models & routing · Lab 22New

Route an alias across models by priority and weight

Put two Foundry models behind one alias, canary a new model version with an 80/20 weighted split, then prefer one model by priority, all without changing the model name that clients send.

Intermediate 25 min PortalCLIBicepTerraform Updated
Source Edit

What you'll do

  • Explain how priority and weight decide which backend model serves a request
  • Put two backend models behind one alias with an 80/20 weighted split
  • Tell which backend model served each request from the response
  • Prefer one backend model by priority, then promote the other without changing clients

Before you start

Tracks: Portal, Azure CLI, Bicep + azd, Terraform

Your team wants to try gpt-6.1-sol, but every app already sends "model": "chat", and nobody wants a big-bang switch. In this lab you put the current model and the new one behind the same alias, send a fifth of the traffic to the new model, count where the requests land, and then use priority to choose which model is preferred. The apps never change the model name they send.

Before you begin

One-time setupThe Setup guide tasks that this lab uses. Mark each one done once, and the labs stop listing it.

Tip

On the Bicep + azd and Terraform tracks, azd up or terraform apply creates its own gateway, both models, the alias, and a key, so you can skip this section. You need only the one-time setup above.

  • Complete Publish a Foundry model and make your first call. You need the gateway, a runtime API key, and the gpt-5-mini backend model. Lab 02 also gave the gateway the Foundry User role on your Foundry account, which covers the new deployment in this lab.
  • Permission to change the gateway, for example Contributor on its resource group, and to create deployments in the Foundry account.
  • Optional: Publish models from any provider behind stable aliases explains aliases in more depth.
  • Run the scripts and samples from the lab's folder, ~/aigateway/labs/22-model-routing.

Set these variables once. Every track and the Verify section reuse them.

# Name your gateway, the Foundry account from lab 02, the two backend models, and the alias that clients call
export RG="<resource-group>"
export GATEWAY="<gateway>"
export FOUNDRY_RG="<foundry-resource-group>"     # resource group of the Foundry account from lab 02
export FOUNDRY_ACCOUNT="<foundry-account-name>"
export PROVIDER="foundry-models"                 # the gateway provider that holds gpt-5-mini
export PRIMARY="gpt-5-mini"                      # the backend model that serves today's traffic
export CANDIDATE="gpt-6.1-sol"                   # the new model to try
export CANDIDATE_VERSION="2026-09-29"
export ALIAS="chat"                              # the name that clients send in "model"
echo "📝 Alias $ALIAS will route between $PRIMARY and $CANDIDATE on gateway $GATEWAY in $RG"

# Read the resource IDs of the gateway and the Foundry account, and the gateway's runtime URL
export GATEWAY_ID="$(az aigateway show -g "$RG" -n "$GATEWAY" --query id -o tsv)"
export FOUNDRY_ID="$(az cognitiveservices account show -g "$FOUNDRY_RG" -n "$FOUNDRY_ACCOUNT" --query id -o tsv)"
export AI_GATEWAY_URL="https://$(az aigateway show -g "$RG" -n "$GATEWAY" --query properties.frontend.defaultHostname -o tsv)"
echo "🆔 Gateway: $GATEWAY_ID"
echo "🆔 Foundry account: $FOUNDRY_ID"
echo "🌐 Runtime URL: $AI_GATEWAY_URL"

# Paste the gateway runtime key from lab 02 without showing it, and don't commit it or write it to a file
printf "AI Gateway API key: " && read -rs AI_GATEWAY_API_KEY && export AI_GATEWAY_API_KEY && echo
echo "🔑 Key: in AI_GATEWAY_API_KEY (${#AI_GATEWAY_API_KEY} characters, not shown)"

The CLI and Bicep tracks of lab 02 name the provider foundry-models. A portal import names it after your Foundry account. To check, list the backend models. The Provider column shows the provider of each one.

# List the backend models on the gateway, with the provider that holds each one
echo "✨ Backend models on $GATEWAY:"
az aigateway model list -g "$RG" --gateway-name "$GATEWAY" -o table

Deploy the candidate model in the same Foundry account as gpt-5-mini. Pick a region where the model is available. Model inference is billed per use.

# Deploy the candidate model next to gpt-5-mini, with 10,000 tokens per minute of Global Standard capacity
az cognitiveservices account deployment create -g "$FOUNDRY_RG" -n "$FOUNDRY_ACCOUNT" \
  --deployment-name "$CANDIDATE" --model-name "$CANDIDATE" --model-version "$CANDIDATE_VERSION" \
  --model-format OpenAI --sku-name GlobalStandard --sku-capacity 10 \
  --query "{name:name, state:properties.provisioningState}" &&
  echo "✨ Model deployment $CANDIDATE is ready in $FOUNDRY_ACCOUNT"

How it works

An alias holds a list of backend models. Each one has a priority and a weight:

SettingValuesDefaultWhat it does
PriorityWhole number from 0 to 1000Lower numbers are preferred. Backend models with the same priority form a group, and requests go to the available group with the lowest number.
WeightWhole number from 1 to 1001Splits the requests within a group in proportion to the weights. Weights count requests, not tokens or cost.

A backend model's Share within priority is its weight divided by the sum of the weights at its priority. Weights 80 and 20 give 80% and 20%; weights 3 and 1 give 75% and 25%. Weights are relative, so 5 and 1 split the requests like 50 and 10. With one backend model, the alias behaves like the single-target aliases of labs 02 and 03.

Priority and weight are two steps of the alias's routing order. The gateway first drops the backend models whose backend condition doesn't match the request. Then it applies priority, auto model selection, and session affinity in the order that the alias sets. Weights choose among the backend models that remain. This lab uses only priority and weight, so the gateway picks the lowest priority group, then a backend model in it by weight.

Pick the values from the job you need to do:

ScenarioPriorityWeight
Canary a new model versionSame for bothSmall share for the new version, for example 95 and 5, then raise it
A/B test two modelsSame for bothEqual, for example 50 and 50
Balance deployments across regions or providersSame for allIn proportion to each deployment's capacity, for example 3 and 1
Prefer purchased capacityProvisioned deployment 0, pay-as-you-go 1Any

A few rules keep every request answerable:

  • Each backend model can appear only once in an alias.
  • All backend models must support at least one interface in common, for example OpenAI Chat Completions. Mixing an Anthropic Messages model with an OpenAI model in one alias fails with Active-priority models must share at least one supported endpoint.
  • The response tells you which backend served the request: the model field holds the backend's model name and version, for example gpt-6.1-sol-2026-09-29, and Foundry models also return an x-ms-served-model header.
Important

Priority sets a preference, not a failover. Without failure rules on the alias, when the preferred backend returns an error such as 429 or 404, the gateway passes that error to the client and doesn't try a lower-priority backend for that request. To retry the backend or switch to the next one, add failure rules to the alias, as in Fail over to a backup model when a backend fails.

Split traffic 80/20 between two models

Register the candidate model in the gateway, then create the chat alias with both models at priority 0 and weights 80 and 20.

Choose your track
1

Import the candidate model

  1. Open the AI Gateway portal and select your gateway in the top bar.
  2. Select ModelsAddModels from Microsoft Foundry.
  3. In Subscriptions and Resource groups, select those of your Foundry account. Keep Managed identity as the Authentication method.
  4. Clear Create an alias for each model: this lab publishes the model through the chat alias.
  5. In Models, select only the gpt-6.1-sol deployment of your Foundry account, then select Add models.
2

Create the chat alias

  1. Select ModelsAddAlias for existing models.
  2. On Basics, set Alias name to chat and Description to Canary: 80/20 between the current and the new model. Select Next.
  3. On Auto model selection (preview), leave Choose a model based on the prompt off. Select Next.
  4. On Select models, select gpt-5-mini and gpt-6.1-sol in the grid. Select Next.
  5. On Session affinity, clear Enable session affinity, which is on by default, so that the weights alone split the requests. Select Next.
  6. On Priorities and weights, keep Priority at 0 for both models. Set Weight to 80 for gpt-5-mini and 20 for gpt-6.1-sol. Share within priority shows 80% and 20%. Select Next.
  7. On Failure handling, keep No custom failure rules configured. Select Next.
  8. On Review, check the shares, then select Create.
Priorities and weights. Both models share priority 0, so the weights split the requests 80/20.
Checkpoint

The alias's Overview shows Session affinity Disabled and Routing order Priorities only. Its Backend models tab shows gpt-5-mini and gpt-6.1-sol, both at priority 0, with weights 80 and 20.

Watch the split

Send 20 requests to chat and count which backend model answered each one. The script sends one request every half second and reads the model field of each response.

# Send 20 requests to the alias, one every half second, and count which backend model answered each one
echo "📊 Counting the backend models that serve $ALIAS, one request every half second"
./infra/count-models.sh "$ALIAS" 20
Text
Requests per backend model:
  16 gpt-5-mini-2025-08-07
   4 gpt-6.1-sol-2026-09-29

In our tests the gateway interleaved the two models, four requests to gpt-5-mini and then one to gpt-6.1-sol, so 20 requests split exactly 16 and 4. Plan around the shares, not the exact order.

To see the header for a single call:

# Send one request to the alias, and print only the x-ms-served-model header of the response
echo "✨ Backend model that served one request to $ALIAS:"
curl -sS -D - -o /dev/null "$AI_GATEWAY_URL/default/models/openai/v1/chat/completions" \
  -H "Authorization: Bearer $AI_GATEWAY_API_KEY" -H "Content-Type: application/json" \
  -d "{\"model\": \"$ALIAS\", \"max_completion_tokens\": 16, \"messages\": [{\"role\": \"user\", \"content\": \"Hi\"}]}" |
  grep -i "^x-ms-served-model"
Checkpoint

About four of every five requests show gpt-5-mini-…, and the rest show gpt-6.1-sol-…. The client sent "model": "chat" every time. Right after you create the alias, the first requests can return HTTP 404 model_not_found for a few seconds; run the count again.

Prefer one model by priority

The canary looks good, but finance wants gpt-5-mini to handle the traffic until the new model is approved. Give gpt-6.1-sol priority 1. It stays in the alias, ready to promote, but requests go to gpt-5-mini.

Choose your track
1

Edit the alias

  1. In ModelsAliases, select chat, then select Edit.
  2. In Priorities and weights, set Priority to 1 for gpt-6.1-sol and set both weights to 1. Share within priority shows 100% for each model: gpt-5-mini at priority 0, and gpt-6.1-sol at priority 1.
  3. Select Save. New requests use the new settings within a few seconds.
Checkpoint

The alias Backend models tab shows gpt-5-mini at priority 0 and gpt-6.1-sol at priority 1.

Verify

1

Read the alias

# Print the alias's backend models, with the priority and the weight of each one
echo "🔖 Alias $ALIAS routes to:"
az aigateway model-alias show -g "$RG" --gateway-name "$GATEWAY" -n "$ALIAS" --query properties.backendModels -o json |
  jq -r '.[] | "\(.resourceId | split("/") | last)  priority \(.priority)  weight \(.weight)"'
2

Count the requests again

# Send 10 requests to the alias and count which backend model answered each one
echo "📊 Counting the backend models that serve $ALIAS, one request every half second"
./infra/count-models.sh "$ALIAS" 10
Checkpoint

All 10 requests show gpt-5-mini-…. gpt-6.1-sol stays in the alias at priority 1. If one of the first requests still shows gpt-6.1-sol-…, the change was still reaching the gateway; run the count again. If some show HTTP 429 rate_limit_exceeded, the gpt-5-mini deployment used up its 10 requests per minute; wait a minute and run the count again.

3

Promote the new model

To finish the rollout, swap the priorities: gpt-6.1-sol at 0, gpt-5-mini at 1. Use the portal, or run the az aigateway model-alias update command from the previous section with $PRIMARY and $CANDIDATE swapped. Run the count again: every request now shows gpt-6.1-sol-…. To roll back, swap them again. Clients never change.

Go further

  • Canary in steps. Move the weights from 95/5 to 80/20 to 50/50 while you watch errors, latency, and cost per model in Monitor usage, cost, and traces. Weights must be at least 1, so to stop sending traffic to a model, give it a higher priority number or remove it from the alias.
  • Prefer provisioned throughput. Put a provisioned deployment at priority 0 and a pay-as-you-go deployment of the same model at priority 1, so purchased capacity is used first. Add a failure rule for 429 so that requests switch to the pay-as-you-go deployment when the provisioned one is saturated, as in Fail over to a backup model when a backend fails.
  • Balance regions or providers. Register the same model from two Foundry accounts, or from Foundry and a custom provider, and give them weights in proportion to their quota.
  • Limit each model and the alias. A token limit on a backend model, as in Throttle tokens per caller with token limits, applies to that model. A limit on the alias applies to every request that calls chat.

Clean up

Keep the gateway, the runtime key, and gpt-5-mini from lab 02. Later labs use them. Remove what you added in this lab: the alias first, then the backend model, then the Foundry deployment.

Choose your track
1

Remove the alias and the candidate model

In ModelsAliases, select More actions on the chat row, then Delete and Delete alias. Requests that use the alias fail immediately; the backend models stay in the gateway. Then delete the gpt-6.1-sol model, and any alias that the import created for it, from Models. Finally, delete the gpt-6.1-sol deployment from your Foundry account, for example with the last command of the CLI track.

azd down and terraform destroy don't touch the gateway, key, and gpt-5-mini from lab 02, because the Bicep + azd and Terraform tracks created their own gateways.

Next steps

Finished the lab?

Mark it complete to track your progress across paths. It's saved only in this browser.