Azure AI Gateway Labs
Documentation Open portal
Models & routing · Lab 28Emerging

Let the gateway choose the model for each request

Put gpt-5-mini and gpt-5.5 behind one alias and let Model Router pick one of them for each prompt. Prove which model served each request, switch the objective between cost and quality, and let callers override it per request.

Intermediate 25 min PortalCLIBicepTerraformNotebook Updated
Notebook Source Edit

What you'll do

  • Explain how Model Router picks a model for each prompt and how the objective trades cost against quality
  • Create an alias with auto model selection over gpt-5-mini and gpt-5.5
  • Prove which model served a request with the x-ms-served-model response header
  • Change the objective and watch the split between the models shift
  • Let callers override the objective or narrow the model pool for one request

Before you start

Tracks: Portal, Azure CLI, Bicep + azd, Terraform, Notebook

Most of the questions that reach your assistant are simple, and gpt-5-mini answers them well for a fraction of the cost of gpt-5.5. A few are hard, and only the larger model gets them right. No app wants to decide which is which. In this lab you put both models behind one alias, auto, and turn on auto model selection: for each request, Microsoft Foundry Model Router reads the prompt and picks the model that fits it, as the alias's objective asks. You prove which model served each prompt, move the whole split toward cost, and then let one caller ask for quality on its own requests. The apps only ever send "model": "auto".

Auto model selection is emerging: it's in preview since the November 6 release, and per-request overrides arrived in the November 9 release.

Before you begin

One-time setupThe Setup guide tasks that this lab uses. Mark each one done once, and the labs stop listing it.

Tip

On the Bicep + azd and Terraform tracks, azd up or terraform apply creates its own gateway, both models, the alias, and a key, so you can skip this section. You need only the one-time setup above.

  • Complete Publish a Foundry model and make your first call. You need the gateway, the app-first-call runtime key, and the gpt-5-mini backend model. Lab 02 also gave the gateway the Foundry User role on your Foundry account, which covers the new deployment in this lab.
  • Recommended: Route an alias across models by priority and weight explains aliases, priority, and weight. This lab adds a new step after them.
  • Permission to change the gateway, for example Contributor on its resource group, and to create deployments in the Foundry account.
  • Run the scripts from the lab's folder, ~/aigateway/labs/28-auto-model-selection. The Bash scripts and snippets need curl and jq, which Cloud Shell has.

Set these variables once. Every track and the Verify section reuse them.

# Name your gateway, the Foundry account from lab 02, the two models, and the alias that clients call
export RG="<resource-group>"
export GATEWAY="<gateway>"
export FOUNDRY_RG="<foundry-resource-group>"     # resource group of the Foundry account from lab 02
export FOUNDRY_ACCOUNT="<foundry-account-name>"
export PROVIDER="foundry-models"                 # the gateway provider that holds gpt-5-mini
export SMALL="gpt-5-mini"                        # the lower-cost model, already published in lab 02
export LARGE="gpt-5.5"                           # the higher-quality model that you deploy in this lab
export LARGE_VERSION="2026-04-24"
export LARGE_BACKEND="gpt-5-5"                   # the gateway's name for gpt-5.5, as the portal import names it
export ALIAS="auto"                              # the name that clients send in "model"
echo "📝 Alias $ALIAS will choose between $SMALL and $LARGE on gateway $GATEWAY in $RG"

# Read the resource IDs of the gateway and the Foundry account, and the gateway's runtime URL
export GATEWAY_ID="$(az aigateway show -g "$RG" -n "$GATEWAY" --query id -o tsv)"
export FOUNDRY_ID="$(az cognitiveservices account show -g "$FOUNDRY_RG" -n "$FOUNDRY_ACCOUNT" --query id -o tsv)"
export AI_GATEWAY_URL="https://$(az aigateway show -g "$RG" -n "$GATEWAY" --query properties.frontend.defaultHostname -o tsv)"
echo "🆔 Gateway: $GATEWAY_ID"
echo "🌐 Runtime URL: $AI_GATEWAY_URL"

# Build the management URL of the gateway's default workspace, and the API version, for the az rest calls
export WS="https://management.azure.com$GATEWAY_ID/workspaces/default"
export API="api-version=2026-11-01"

# Read the app-first-call key from the gateway, so that it never shows on screen or lands in a file
export AI_GATEWAY_API_KEY="$(az aigateway api-key list-secrets -g "$RG" --gateway-name "$GATEWAY" -n app-first-call \
  --query primaryKey -o tsv)"
echo "🔑 Key: in AI_GATEWAY_API_KEY (${#AI_GATEWAY_API_KEY} characters, not shown)"

# Keep a jq filter that prints the alias's auto model selection: each backend model, the routing order, the objective,
# and what callers may override
export AUTO='.properties
  | (.backendModels[] | "\(.resourceId | split("/") | last)  selector \(.selectorModelName // "none")  priority \(.priority)  weight \(.weight)"),
    "Routing order: \((.selectionOrder // ["priority"]) | join(", "))",
    "Objective: \(.autoModelSelection.objective // "off")",
    "Caller overrides: \((.autoModelSelection.allowedRequestOverrides // []) | if length == 0 then "none" else join(", ") end)"'

The CLI and Bicep tracks of lab 02 name the provider foundry-models. A portal import names it after your Foundry account. To check, list the backend models. The Provider column shows the provider of each one.

# List the backend models on the gateway, with the provider that holds each one
echo "✨ Backend models on $GATEWAY:"
az aigateway model list -g "$RG" --gateway-name "$GATEWAY" -o table

Deploy gpt-5.5 in the same Foundry account as gpt-5-mini. Pick a region where the model is available. Model inference is billed per use, and gpt-5.5 costs more per token than gpt-5-mini: that difference is what auto model selection saves.

# Deploy gpt-5.5 next to gpt-5-mini, with 20,000 tokens and 20 requests per minute of Global Standard capacity
az cognitiveservices account deployment create -g "$FOUNDRY_RG" -n "$FOUNDRY_ACCOUNT" \
  --deployment-name "$LARGE" --model-name "$LARGE" --model-version "$LARGE_VERSION" \
  --model-format OpenAI --sku-name GlobalStandard --sku-capacity 20 \
  --query "{name:name, state:properties.provisioningState}" &&
  echo "✨ Model deployment $LARGE is ready in $FOUNDRY_ACCOUNT"

How it works

Lab 22 picked a backend model with priority and weight, and lab 27 added backend conditions and session affinity. Auto model selection adds a step that reads the prompt itself. You turn it on in an alias with four settings:

SettingAlias fieldPortalWhat it does
Auto model selectionautoModelSelectionAuto model selection (preview)Choose a model based on the promptAsks Model Router to choose among the alias's models for each request.
ObjectiveautoModelSelection.objectiveObjectiveWhat Model Router favors: Balanced, Cost, or Quality.
Selector modelbackendModels[].selectorModelNameSelect modelsSelector modelThe unversioned model name that Model Router knows each backend model by, such as gpt-5-mini. It names the model that the deployment serves, not the backend model's name in the gateway.
Routing orderselectionOrderRouting orderWhen Model Router runs. ["priority", "autoModelSelection"], Priorities before automatic model selection, keeps the best available priority group, then lets Model Router choose a model in it.

For each request, the gateway first drops the backend models whose condition is false. Then it applies the routing order: here, priority keeps the backend models of the best available priority group, and Model Router reads the prompt and picks one of their selector models, as the objective asks. If several backend models serve that model, weight or session affinity picks one. Then the gateway calls it, and three response headers tell you what happened: x-ms-served-model names the model and version that answered, x-ms-model-router-model names the model that Model Router picked, as name@version, and x-ms-model-router-mode names the objective that it used.

The portal describes the three objectives:

ObjectivePortal hintWhat you see in this lab
BalancedBalance quality and cost across simpler and harder requests.Simple prompts mostly go to gpt-5-mini, hard ones to gpt-5.5.
QualityFavor the highest-quality eligible model, regardless of cost.Every prompt goes to gpt-5.5.
CostFavor lower-cost models, with a quality tradeoff on harder requests.Every prompt goes to gpt-5-mini, even the hard ones.

Three rules shape what you can put behind an auto alias:

  • Model Router supports a defined set of models. In the portal, Select modelsView supported models opens the Model Router catalog page, and the portal warns Some models aren't supported for auto model selection. when you pick another one. The list includes OpenAI models such as gpt-5-nano, gpt-5-mini, gpt-5, gpt-5.4, and gpt-5.5, and models from other providers, such as Claude, Grok, DeepSeek, and Llama models. This lab uses gpt-5-mini and gpt-5.5, because gpt-6.1-sol, the larger model of labs 22 and 27, isn't on the list when we write this.
  • Every backend model must share an API endpoint, and requests must be text. Model Router rejects embeddings and images, and a new selection can't continue a conversation that a provider stored, such as a Responses request with previous_response_id. Send the whole conversation in each request.
  • Callers can't change the selection unless you allow it. Per-request overrides are off by default. You turn them on in the last section.

Auto model selection is in preview. The portal notes that model selection has no additional charge during the preview, that the backend models' usage is still billed, and that Model Router pricing applies once the feature is generally available.

Create the auto alias

Register gpt-5.5 in the gateway, then create the auto alias with both models at the same priority, so that priority keeps both and Model Router chooses between them. Start with the Balanced objective and no per-request overrides.

Choose your track
1

Import gpt-5.5

  1. Open the AI Gateway portal and select your gateway in the top bar.

  2. Select ModelsAddModels from Microsoft Foundry.

  3. In Resource groups, select the resource group of your Foundry account. Keep Managed identity as the Authentication method, and clear Create an alias for each model.

  4. In Models, the list selects every Foundry account it finds. Make sure only your Foundry account is selected. Its row reads This model provider already exists in your gateway. 1 new model is available. Select Add models, then Close.

    The gateway already has gpt-5-mini, so the import adds only gpt-5.5.
Checkpoint

ModelsBackend models lists gpt-5-mini and gpt-5.5. The portal names the new backend model gpt-5-5 and shows its display name, gpt-5.5.

2

Create the auto alias

  1. Select ModelsAddAlias for existing models.

    An alias is the callable name. Its backend models are the deployments that can serve it.
  2. On Basics, set Alias name to auto and Description to Model Router picks gpt-5-mini or gpt-5.5 for each prompt. Select Next.

  3. On Auto model selection (preview), select Choose a model based on the prompt. Keep Objective at Balanced, and leave every option under Per-request overrides cleared. Select Next.

    The objective sets what Model Router favors. Per-request overrides stay off until you allow them.
  4. On Select models, select gpt-5-mini and gpt-5.5. Check that the Selector model column shows gpt-5-mini and gpt-5.5, and pick them from the list if it doesn't. Select Next.

    The selector model names the model that each backend serves, which is how Model Router knows it.
  5. On Session affinity, clear Enable session affinity, so that every request gets its own choice. Select Next.

  6. On Priorities and weights, keep Priority 0 and Weight 1 for both models. Select Next.

  7. On Routing order, keep Priorities before automatic model selection. Select Next.

    Priority keeps your preferred deployment group first. Model Router then picks a model inside it.
  8. On Failure handling, keep the defaults, and select Next.

  9. On Review, check that Objective reads Balanced, Per-request overrides reads Not allowed, Session affinity reads Disabled, and Models for selection reads gpt-5.5 (1 backend model), gpt-5-mini (1 backend model). Select Create.

    Models for selection counts the backend models behind each selector model.
Checkpoint

The auto alias opens on Overview. Auto model selection (preview) reads On, Auto model selection objective reads Balanced (balance quality and cost across simpler and harder requests), Per-request overrides reads Not allowed, and Routing order reads Priorities before automatic model selection.

Test in catalog opens the alias in the playground.

Send a simple and a hard prompt

Send two prompts to auto: a yes-or-no question and a proof. Each answer stops at 1,000 output tokens to keep the run short and cheap, which doesn't change the model that Model Router picks. The commands print only the three response headers of auto model selection. A new alias takes up to a minute to reach every gateway instance: until then, requests return HTTP 404 with 'auto' is not a known model alias.

# Send a simple and a hard prompt to the alias, and print the model that served each one, the model that Model Router
# picked, and the objective that it used
for PROMPT in "Is 17 a prime number? Answer yes or no." \
  "Prove that the square root of 2 is irrational, then generalize the proof to show that the square root of any prime number is irrational."; do
  echo "📊 $PROMPT"
  jq -cn --arg m "$ALIAS" --arg p "$PROMPT" '{model: $m, max_completion_tokens: 1000, messages: [{role: "user", content: $p}]}' |
    curl -sS -D - -o /dev/null "$AI_GATEWAY_URL/default/models/openai/v1/chat/completions" \
      -H "Authorization: Bearer $AI_GATEWAY_API_KEY" -H "Content-Type: application/json" -d @- |
    grep -iE "^x-ms-(served-model|model-router-model|model-router-mode):"
done
Text
📊 Is 17 a prime number? Answer yes or no.
x-ms-served-model: gpt-5-mini-2025-08-07
x-ms-model-router-model: gpt-5-mini@2025-08-07
x-ms-model-router-mode: balanced
📊 Prove that the square root of 2 is irrational, then generalize the proof to show that the square root of any prime number is irrational.
x-ms-served-model: gpt-5.5-2026-04-24
x-ms-model-router-model: gpt-5.5@2026-04-24
x-ms-model-router-mode: balanced

The same alias sent the yes-or-no question to gpt-5-mini and the proof to gpt-5.5, without any hint from the caller. x-ms-served-model names the model and version that answered, and the response body's model field agrees. x-ms-model-router-model shows that Model Router made that choice, and x-ms-model-router-mode that it used the alias's Balanced objective.

On the portal track, you can see the same in the playground. On the alias's Overview, select Test in catalog, then send both prompts. The line under each answer shows the time, the tokens, and the model that served it.

The playground names the model that served each answer.
Checkpoint

The simple prompt shows x-ms-served-model: gpt-5-mini-2025-08-07, and the hard one x-ms-served-model: gpt-5.5-2026-04-24, both with x-ms-model-router-mode: balanced. If you see HTTP 404 with model_not_found, wait a minute, and send them again.

Run a set of prompts

infra/prompts.json holds five simple prompts and five hard ones: proofs, a scheduling puzzle, and a system design question. The script sends them one at a time, then prints the model that served each one with its token usage, and the totals per model.

# Send the ten prompts to the alias with its own objective, and count the requests and tokens per model
echo "📊 Which model serves each prompt with the alias's objective"
./infra/route-prompts.sh "$ALIAS"
Text
Sending 10 prompts with "model": "auto" and the alias's objective
  #  kind    served model              prompt  output  text
  1  simple  gpt-5-mini-2025-08-07         18      10  Is 17 a prime number? Answer yes or no.
  2  simple  gpt-5-mini-2025-08-07         16      94  Translate 'thank you very much' into French.
  3  simple  gpt-5-mini-2025-08-07         14      10  What is 12 times 12?
  4  simple  gpt-5.5-2026-04-24            12      13  What does HTTP stand for?
  5  simple  gpt-5.5-2026-04-24            14      13  Which planet is closest to the sun?
  6  hard    gpt-5.5-2026-04-24            36     746  Prove that the square root of 2 is irrational, the
  7  hard    gpt-5.5-2026-04-24            86     719  A train leaves at 09:40 and travels 315 km at 84 k
  8  hard    gpt-5.5-2026-04-24            86    1000  Let f be a continuous function on [0,1] with f(0)=
  9  hard    gpt-5.5-2026-04-24            54    1000  Design a sharded rate limiter for 50 million API k
 10  hard    gpt-5.5-2026-04-24            40    1000  Find all integer solutions of x^2 - 7y^2 = 1 with 

Requests and tokens per model:
gpt-5-mini-2025-08-07      3 requests      48 prompt tokens     114 output tokens
gpt-5.5-2026-04-24         7 requests     328 prompt tokens    4491 output tokens

With Balanced, Model Router sent every hard prompt to gpt-5.5, and three of the five simple ones to gpt-5-mini. It sent the two short factual questions to gpt-5.5 too: Balanced weighs quality as well as cost, and the answer is so short that the larger model costs little. Its choices for borderline prompts can change from one run to the next. The hard prompts use most of the output tokens, so they're where the larger model's price matters.

Checkpoint

Every row shows an answer from gpt-5-mini-2025-08-07 or gpt-5.5-2026-04-24, the five hard rows show gpt-5.5-2026-04-24, and the totals list both models. A HTTP 429, retrying line means that a deployment reached its requests per minute; the script waits and sends the prompt again.

Switch the objective to cost

Your finance team asks you to cut the cost of this assistant, and accepts weaker answers on hard questions. Change the alias's objective to Cost. The apps don't change.

Choose your track
1

Switch the objective

  1. In ModelsAliases, open auto and select Edit. The alias opens on one page, with a section for each step.
  2. In Auto model selection (preview)Objective, select Cost.
  3. Select Save. New requests use the new objective within a minute.
Checkpoint

The alias Overview shows Auto model selection objective Cost (favor lower-cost models, with a quality tradeoff on harder requests).

Run the same prompts again:

# Send the ten prompts again, now with the Cost objective
echo "📊 Which model serves each prompt with the Cost objective"
./infra/route-prompts.sh "$ALIAS"
Text
Sending 10 prompts with "model": "auto" and the alias's objective
  #  kind    served model              prompt  output  text
  1  simple  gpt-5-mini-2025-08-07         18     138  Is 17 a prime number? Answer yes or no.
  2  simple  gpt-5-mini-2025-08-07         16     153  Translate 'thank you very much' into French.
  3  simple  gpt-5-mini-2025-08-07         14      10  What is 12 times 12?
  4  simple  gpt-5-mini-2025-08-07         12     102  What does HTTP stand for?
  5  simple  gpt-5-mini-2025-08-07         14      95  Which planet is closest to the sun?
  6  hard    gpt-5-mini-2025-08-07         36    1000  Prove that the square root of 2 is irrational, the
  7  hard    gpt-5-mini-2025-08-07         86    1000  A train leaves at 09:40 and travels 315 km at 84 k
  8  hard    gpt-5-mini-2025-08-07         86    1000  Let f be a continuous function on [0,1] with f(0)=
  9  hard    gpt-5-mini-2025-08-07         54    1000  Design a sharded rate limiter for 50 million API k
 10  hard    gpt-5-mini-2025-08-07         40    1000  Find all integer solutions of x^2 - 7y^2 = 1 with 

Requests and tokens per model:
gpt-5-mini-2025-08-07     10 requests     376 prompt tokens    5498 output tokens

Every prompt now goes to gpt-5-mini, the hard ones too. That's the trade-off that the portal hint warns about: the bill drops, and the hard answers come from the smaller model. gpt-5-mini also used the whole 1,000-token cap on every hard prompt, so some of its answers stop before the end. Compare its answers to the hard prompts with the earlier run's before you pick this objective for a real workload.

Checkpoint

All ten rows show gpt-5-mini-2025-08-07, and the totals list only that model. If a few rows still show gpt-5.5, the change hadn't reached every gateway instance yet: wait a minute, and run the script again.

Let callers choose per request

One app, a tutoring tool, needs the best answer every time, even though the alias now favors cost. Instead of a second alias, let callers send an auto_model_selection field in the request body. Its objective overrides the alias's objective, and its models narrows the pool to some of the alias's selector models. Both are off by default, so try one first:

# Ask for the Quality objective in the request body, and print the response and its status code
jq -cn --arg m "$ALIAS" '{model: $m, max_completion_tokens: 1000, auto_model_selection: {objective: "quality"},
    messages: [{role: "user", content: "Prove that the square root of 2 is irrational."}]}' |
  curl -sS "$AI_GATEWAY_URL/default/models/openai/v1/chat/completions" -w '\nHTTP %{http_code}\n' \
    -H "Authorization: Bearer $AI_GATEWAY_API_KEY" -H "Content-Type: application/json" -d @-
Text
{"error":{"code":"request_override_not_allowed","message":"The Alias does not permit every supplied auto_model_selection field.","param":"auto_model_selection","type":"invalid_request_error"}}
HTTP 403

The gateway refuses the request with HTTP 403 before it reaches a model. Allow both overrides on the alias:

Choose your track
1

Allow per-request overrides

  1. Open auto and select Edit.

  2. Under Per-request overrides, select Allow callers to override the objective and Allow callers to narrow the pool of eligible models. Keep the Cost objective.

  3. Select Save.

    The two filter options aren't caller overrides. They're saved alias settings, described in Go further.
Checkpoint

The alias Overview shows Per-request overrides Objective, Eligible model pool.

Run the prompts again, and this time ask for Quality in each request. The script sends "auto_model_selection": {"objective": "Quality"}; the values aren't case-sensitive.

# Send the ten prompts with the Quality objective in each request, while the alias keeps Cost
echo "📊 Which model serves each prompt when the caller asks for Quality"
./infra/route-prompts.sh "$ALIAS" Quality
Text
Sending 10 prompts with "model": "auto" and the Quality objective
  #  kind    served model              prompt  output  text
  1  simple  gpt-5.5-2026-04-24            18      16  Is 17 a prime number? Answer yes or no.
  2  simple  gpt-5.5-2026-04-24            16      17  Translate 'thank you very much' into French.
  3  simple  gpt-5.5-2026-04-24            14      21  What is 12 times 12?
  4  simple  gpt-5.5-2026-04-24            12      13  What does HTTP stand for?
  5  simple  gpt-5.5-2026-04-24            14      13  Which planet is closest to the sun?
  6  hard    gpt-5.5-2026-04-24            36     764  Prove that the square root of 2 is irrational, the
  7  hard    gpt-5.5-2026-04-24            86     880  A train leaves at 09:40 and travels 315 km at 84 k
  8  hard    gpt-5.5-2026-04-24            86    1000  Let f be a continuous function on [0,1] with f(0)=
  9  hard    gpt-5.5-2026-04-24            54    1000  Design a sharded rate limiter for 50 million API k
 10  hard    gpt-5.5-2026-04-24            40    1000  Find all integer solutions of x^2 - 7y^2 = 1 with 

Requests and tokens per model:
gpt-5.5-2026-04-24        10 requests     376 prompt tokens    4724 output tokens

Every prompt of this caller goes to gpt-5.5, while other callers keep the alias's Cost objective. Now narrow the pool instead: ask for Quality, but only from gpt-5-mini. models takes selector model names, not backend model names.

# Ask for the Quality objective but only gpt-5-mini, and print the auto model selection headers of the hard prompt
jq -cn --arg m "$ALIAS" --arg small "$SMALL" '{model: $m, max_completion_tokens: 1000,
    auto_model_selection: {objective: "quality", models: [$small]},
    messages: [{role: "user", content: "Prove that the square root of 2 is irrational."}]}' |
  curl -sS -D - -o /dev/null "$AI_GATEWAY_URL/default/models/openai/v1/chat/completions" \
    -H "Authorization: Bearer $AI_GATEWAY_API_KEY" -H "Content-Type: application/json" -d @- |
  grep -iE "^x-ms-(served-model|model-router-model|model-router-mode):"
Text
x-ms-served-model: gpt-5-mini-2025-08-07
x-ms-model-router-model: gpt-5-mini@2025-08-07

The pool held one model, so gpt-5-mini answered the proof even though the request asked for Quality. With a single model to choose from, the response has no x-ms-model-router-mode header.

On the portal track, the playground now shows an objective list and a model list in the message composer. Select Quality and All models, and send the proof prompt: gpt-5.5 answers.

When the alias allows overrides, the composer can set the objective and the model pool for each message.
Checkpoint

With the Quality override, every row shows gpt-5.5-2026-04-24. If the first rows show HTTP 403 request_override_not_allowed, the alias change hadn't reached every gateway instance yet: wait a minute, and run the script again. With models: ["gpt-5-mini"], the proof shows x-ms-served-model: gpt-5-mini-2025-08-07. A pool with a name that isn't a selector model of the alias, such as gpt-5-5, returns HTTP 400 with auto_model_selection.models must contain unique names from this Alias's configured selector Model pool.

Verify

1

Read the alias

# Read the alias, and print its auto model selection
echo "🔖 Alias $ALIAS:"
az aigateway model-alias show -g "$RG" --gateway-name "$GATEWAY" -n "$ALIAS" | jq -r "$AUTO"
Checkpoint

Both backend models show their selector names at priority 0 weight 1, followed by Routing order: priority, autoModelSelection, Objective: Cost, and Caller overrides: model, objective.

2

Send the hard prompt with each objective

# Send the proof prompt once with each objective in the request, and print the model that served it
for OBJECTIVE in cost balanced quality; do
  jq -cn --arg m "$ALIAS" --arg o "$OBJECTIVE" '{model: $m, max_completion_tokens: 1000,
      auto_model_selection: {objective: $o},
      messages: [{role: "user", content: "Prove that the square root of 2 is irrational."}]}' |
    curl -sS -D - -o /dev/null "$AI_GATEWAY_URL/default/models/openai/v1/chat/completions" \
      -H "Authorization: Bearer $AI_GATEWAY_API_KEY" -H "Content-Type: application/json" -d @- |
    grep -i "^x-ms-served-model" | sed "s/^/✨ $OBJECTIVE: /"
done
Checkpoint

cost shows gpt-5-mini-2025-08-07, and quality shows gpt-5.5-2026-04-24. balanced usually shows gpt-5.5-2026-04-24 for a proof.

Go further

  • Let Model Router choose before priority. Add a second deployment of gpt-5-mini in another region at priority 1, with the same selector model. With Priorities before automatic model selection, Model Router chooses among the priority 0 models only, and the backup serves only when they're unavailable. With Automatic model selection before priorities (["autoModelSelection", "priority"]), Model Router chooses the model across every priority group first, then the gateway uses that model's best available priority group.
  • Keep a conversation on one model. Add session affinity, as in lab 27, and pick a routing order that includes it, such as Session affinity, then priorities, then automatic model selection (["sessionBackendPreference", "priority", "autoModelSelection"]). When affinity comes first, a request can reuse its session's backend model without another Model Router call, which keeps prompt caches warm.
  • Filter models for tools and request options. Filter models for function-tool compatibility and Filter models for request-parameter compatibility are saved alias settings, not caller fields. Over the API, they're the allowedRequestOverrides values filterModelsForToolCompatibility and filterModelsForParameterCompatibility. They keep Model Router to the models that can handle a request's function tools or its native Chat Completions and Responses options.
  • Cap the spend too. Model Router lowers the cost per request, but doesn't cap it. Add a budget per caller, as in Enforce cost budgets with per-caller overrides.
  • Watch the split in production. Follow the requests and token usage of the alias over time, as in Monitor usage, cost, and traces.

Clean up

Keep the gateway, the runtime key, and gpt-5-mini from lab 02. Later labs use them. Remove what you added in this lab: the alias first, then the backend model, then the Foundry deployment.

Choose your track
1

Remove the alias and gpt-5.5

Open auto in ModelsAliases and select Delete. Requests that use the alias fail immediately; the backend models stay in the gateway. Then delete gpt-5-5 from ModelsBackend models. Finally, delete the gpt-5.5 deployment from your Foundry account, for example with the last command of the CLI track.

azd down and terraform destroy don't touch the gateway, key, and gpt-5-mini from lab 02, because the Bicep + azd and Terraform tracks created their own gateways.

Next steps

Finished the lab?

Mark it complete to track your progress across paths. It's saved only in this browser.