Azure AI Gateway Labs
Documentation Open portal
Models & routing · Lab 27New

Route alias requests with conditions and session affinity

Send the requests that carry a research header to a larger model with a CEL backend condition, then split traffic between two models and keep every conversation on one of them with session affinity, all behind one alias.

Intermediate 30 min PortalCLIBicepTerraform Updated
Source Edit

What you'll do

  • Explain how backend conditions, session affinity, priority, and weight combine to pick a backend model
  • Send the requests that carry a header to a larger model with a CEL backend condition
  • Keep every request of a session on the same backend model with a session key expression
  • Choose whether session affinity or priority decides first with the routing order

Before you start

Tracks: Portal, Azure CLI, Bicep + azd, Terraform

Your research team needs gpt-6.1-sol for deep analysis, but gpt-5-mini is good enough for everyone else, and every app already sends "model": "chat". In this lab you give the larger model a backend condition, so that only requests with the header x-workload: research reach it. Then you split the traffic between both models and add session affinity, so that every turn of a conversation stays on the same model, which keeps answers consistent and prompt caches warm. The apps never change the model name they send.

Before you begin

One-time setupThe Setup guide tasks that this lab uses. Mark each one done once, and the labs stop listing it.

Tip

On the Bicep + azd and Terraform tracks, azd up or terraform apply creates its own gateway, both models, the alias, and a key, so you can skip this section. You need only the one-time setup above.

  • Complete Publish a Foundry model and make your first call. You need the gateway, the app-first-call runtime key, and the gpt-5-mini backend model. Lab 02 also gave the gateway the Foundry User role on your Foundry account, which covers the new deployment in this lab.
  • Recommended: Route an alias across models by priority and weight explains priority and weight. This lab builds on them. If you kept lab 22's chat alias, this lab replaces it.
  • Permission to change the gateway, for example Contributor on its resource group, and to create deployments in the Foundry account.
  • Run the scripts from the lab's folder, ~/aigateway/labs/27-conditional-routing. The Bash scripts and snippets need curl and jq, which Cloud Shell has.

Set these variables once. Every track and the Verify section reuse them.

# Name your gateway, the Foundry account from lab 02, the two backend models, and the alias that clients call
export RG="<resource-group>"
export GATEWAY="<gateway>"
export FOUNDRY_RG="<foundry-resource-group>"     # resource group of the Foundry account from lab 02
export FOUNDRY_ACCOUNT="<foundry-account-name>"
export PROVIDER="foundry-models"                 # the gateway provider that holds gpt-5-mini
export GENERAL="gpt-5-mini"                      # the backend model for everyday requests
export RESEARCH="gpt-6.1-sol"                    # the larger model for research requests
export RESEARCH_VERSION="2026-09-29"
export ALIAS="chat"                              # the name that clients send in "model"
echo "📝 Alias $ALIAS will route between $GENERAL and $RESEARCH on gateway $GATEWAY in $RG"

# Read the resource IDs of the gateway and the Foundry account, and the gateway's runtime URL
export GATEWAY_ID="$(az aigateway show -g "$RG" -n "$GATEWAY" --query id -o tsv)"
export FOUNDRY_ID="$(az cognitiveservices account show -g "$FOUNDRY_RG" -n "$FOUNDRY_ACCOUNT" --query id -o tsv)"
export AI_GATEWAY_URL="https://$(az aigateway show -g "$RG" -n "$GATEWAY" --query properties.frontend.defaultHostname -o tsv)"
echo "🆔 Gateway: $GATEWAY_ID"
echo "🌐 Runtime URL: $AI_GATEWAY_URL"

# Build the management URL of the gateway's default workspace, and the API version, for the az rest calls
export WS="https://management.azure.com$GATEWAY_ID/workspaces/default"
export API="api-version=2026-11-01"

# Read the app-first-call key from the gateway, so that it never shows on screen or lands in a file
export AI_GATEWAY_API_KEY="$(az aigateway api-key list-secrets -g "$RG" --gateway-name "$GATEWAY" -n app-first-call \
  --query primaryKey -o tsv)"
echo "🔑 Key: in AI_GATEWAY_API_KEY (${#AI_GATEWAY_API_KEY} characters, not shown)"

# Keep a jq filter that prints the alias's routing: each backend model with its priority, weight, and condition
export ROUTES='.properties
  | (.backendModels[] | "\(.resourceId | split("/") | last)  priority \(.priority)  weight \(.weight)  "
      + (if .when then "when \(.when)" else "always included" end)),
    "Routing order: \((.selectionOrder // ["priority"]) | join(", "))",
    "Session key: \(.sessionBackendPreference.key // "none")"'

The CLI and Bicep tracks of lab 02 name the provider foundry-models. A portal import names it after your Foundry account. To check, list the backend models. The Provider column shows the provider of each one.

# List the backend models on the gateway, with the provider that holds each one
echo "✨ Backend models on $GATEWAY:"
az aigateway model list -g "$RG" --gateway-name "$GATEWAY" -o table

Deploy the research model in the same Foundry account as gpt-5-mini. If you kept it from lab 22, the command leaves it as it is. Pick a region where the model is available. Model inference is billed per use.

# Deploy the research model next to gpt-5-mini, with 50,000 tokens and 50 requests per minute of Global Standard capacity
az cognitiveservices account deployment create -g "$FOUNDRY_RG" -n "$FOUNDRY_ACCOUNT" \
  --deployment-name "$RESEARCH" --model-name "$RESEARCH" --model-version "$RESEARCH_VERSION" \
  --model-format OpenAI --sku-name GlobalStandard --sku-capacity 50 \
  --query "{name:name, state:properties.provisioningState}" &&
  echo "✨ Model deployment $RESEARCH is ready in $FOUNDRY_ACCOUNT"

How it works

Lab 22 picked a backend model with priority and weight. Two more settings run before them:

SettingAlias fieldPortalWhat it does
Backend conditionbackendModels[].whenSelect models…Set conditionA CEL expression that returns true or false for each request. false removes the backend model from this request's choices. A backend model without a condition is always included.
Session keysessionBackendPreference.keySession affinitySession key expressionA CEL expression that returns a string. Requests with the same key prefer the same backend model, from the first request on. An empty string means no affinity.
Routing orderselectionOrderRouting orderWhether session affinity runs before or after priority. The default, ["priority"], has no affinity.

For each request, the gateway first drops the backend models whose condition is false. Then session affinity and priority narrow the rest in the routing order, and weight splits what's left. If every backend model is excluded, the request fails before it reaches a provider.

Conditions and session keys are CEL expressions over the request. request.headers maps each header name, in lowercase, to a list of values, so read the first value with [0]. Other roots include authorization.key.resource.name (the runtime key's name) and authorization.jwt.claims. Over the API, in Bicep, and in Terraform, wrap each expression in @cel('…'). The portal adds the wrapper for you.

Three habits keep expressions safe:

  • Check that a header exists before you read it. 'x-workload' in request.headers && … returns false when the header is missing. In our tests, reading a missing header failed the request with HTTP 500.
  • Compare exact values. Comparisons are case-sensitive: x-workload: Research doesn't match 'research'.
  • Route with headers, don't authorize with them. Any caller with a key can send x-workload: research. To limit the larger model to some apps, test the key name or a token claim instead, as in Go further.

Session affinity matters for two reasons. Providers cache the long prompt prefix of a conversation per deployment, so when every turn lands on the same backend model, cache hits cut latency and token cost. A weighted split alone can move consecutive turns between models and miss the cache. And a Responses API conversation that continues with previous_response_id must reach the backend that stored the earlier response: the gateway keeps those continuations on their owning backend. Affinity is best effort, not guaranteed session continuity. If the preferred backend is excluded or unavailable, the other settings decide, and adding or removing backend models can change which backend a key maps to. Keep secrets out of session keys.

Route research requests to the larger model

Register gpt-6.1-sol in the gateway, then create the chat alias. gpt-6.1-sol gets priority 0 and the research condition, so research requests prefer it. gpt-5-mini gets priority 1 and no condition, so it serves every other request.

Choose your track
1

Import the research model

  1. Open the AI Gateway portal and select your gateway in the top bar.
  2. Select ModelsAddModels from Microsoft Foundry.
  3. In Resource groups, select the resource group of your Foundry account. Keep Managed identity as the Authentication method, and clear Create an alias for each model.
  4. In Models, the list selects every Foundry account it finds. Make sure only your Foundry account is selected, then select Add models. The banner reads 1 model(s) were created. No aliases were created.: the gateway already has gpt-5-mini, so it adds only gpt-6.1-sol. Select Close.
Checkpoint

ModelsBackend models lists gpt-5-mini and gpt-6.1-sol. The portal names the new backend model gpt-6-1-sol and shows its display name, gpt-6.1-sol.

2

Create the chat alias with a condition

If you kept lab 22's chat alias, delete it in ModelsAliases first, or open it, select Edit, and make the same changes on one page.

  1. Select ModelsAddAlias for existing models.

  2. On Basics, set Alias name to chat and Description to Research requests go to the larger model. Select Next.

  3. On Auto model selection (preview), leave the option cleared, and select Next.

  4. On Select models, select gpt-5-mini and gpt-6.1-sol. In the gpt-6.1-sol row, select …Set condition.

    Each backend model row has a Set condition action. The Condition column shows Always included until you set one.
  5. In Backend condition for gpt-6.1-sol, select Include conditionally, and enter this Condition expression, without the @cel() wrapper:

    Text
    'x-workload' in request.headers && request.headers['x-workload'][0] == 'research'

    The editor checks the syntax as you type. An unbalanced quote or parenthesis shows Check the expression's parentheses, strings, and comments. It doesn't check that a header exists, so keep the in guard.

    Select Apply. The Condition column shows Condition set for gpt-6.1-sol and Always included for gpt-5-mini. Select Next.

    Return true to include the backend model for a request, false to exclude it.
  6. On Session affinity, clear Enable session affinity. You turn it on in the next section. Select Next.

  7. On Priorities and weights, set Priority to 0 for gpt-6.1-sol and 1 for gpt-5-mini. Keep both weights at 1. Select Next.

    The conditional backend model gets the preferred priority. Requests that it can't serve fall through to priority 1.
  8. On Failure handling, keep the defaults, and select Next.

  9. On Review, check that Session affinity reads Disabled, then select Create.

Checkpoint

The chat alias opens. Its Backend models tab shows gpt-6.1-sol with Condition set at priority 0, and gpt-5-mini with Always included at priority 1.

Watch the conditions

Send a few requests to chat without a header, with x-workload: research, and with x-workload: Research, and count which backend model answered each one. The script reads the model field of each response.

# Send 4 requests without a workload header, then 4 with the research header, then 4 with a capitalized value
echo "📊 Counting the backend models that serve $ALIAS for each header"
./infra/count-models.sh "$ALIAS" 4
./infra/count-models.sh "$ALIAS" 4 "x-workload: research"
./infra/count-models.sh "$ALIAS" 4 "x-workload: Research"
Text
📊 Counting the backend models that serve chat for each header
Sending 4 requests with "model": "chat"
  1  gpt-5-mini-2025-08-07
  2  gpt-5-mini-2025-08-07
  3  gpt-5-mini-2025-08-07
  4  gpt-5-mini-2025-08-07

Requests per backend model:
   4 gpt-5-mini-2025-08-07
Sending 4 requests with "model": "chat" and the header x-workload: research
  1  gpt-6.1-sol-2026-09-29
  2  gpt-6.1-sol-2026-09-29
  3  gpt-6.1-sol-2026-09-29
  4  gpt-6.1-sol-2026-09-29

Requests per backend model:
   4 gpt-6.1-sol-2026-09-29
Sending 4 requests with "model": "chat" and the header x-workload: Research
  1  gpt-5-mini-2025-08-07
  2  gpt-5-mini-2025-08-07
  3  gpt-5-mini-2025-08-07
  4  gpt-5-mini-2025-08-07

Requests per backend model:
   4 gpt-5-mini-2025-08-07

Without the header, the condition is false, so gpt-6.1-sol is excluded and gpt-5-mini serves the request at priority 1. With the header, both models are included, and priority prefers gpt-6.1-sol. Research with a capital R doesn't match. Foundry models also return the backend model in an x-ms-served-model header:

# Send one research request to the alias, and print only the x-ms-served-model header of the response
echo "✨ Backend model that served one research request to $ALIAS:"
curl -sS -D - -o /dev/null "$AI_GATEWAY_URL/default/models/openai/v1/chat/completions" \
  -H "Authorization: Bearer $AI_GATEWAY_API_KEY" -H "Content-Type: application/json" -H "x-workload: research" \
  -d "{\"model\": \"$ALIAS\", \"max_completion_tokens\": 16, \"messages\": [{\"role\": \"user\", \"content\": \"Hi\"}]}" |
  grep -i "^x-ms-served-model"
Checkpoint

Requests without the header, or with Research, show gpt-5-mini-…. Requests with x-workload: research show gpt-6.1-sol-…, and the header reads x-ms-served-model: gpt-6.1-sol-2026-09-29. Right after you create or change the alias, the first requests can return HTTP 404 model_not_found or use the old routing for a few seconds. An HTTP 429 line means that a deployment reached its requests per minute. In both cases, wait a minute and run the count again.

Keep each session on one backend

The research pilot is over, and the team wants to compare both models on live traffic with a 50/50 split. Chat apps send many turns per conversation, though, and a split by weight alone moves turns between models. Remove the condition, give both models priority 0, and turn on session affinity keyed on a session-id header, so that every conversation stays on one model.

Choose your track
1

Edit the alias

  1. In ModelsAliases, open chat and select Edit. The alias opens on one page, with a section for each step.

  2. In Select models, in the gpt-6.1-sol row, select …Edit condition. Clear Include conditionally, and select Apply. The Condition column shows Always included for both models.

  3. In Session affinity, select Enable session affinity. The Session key expression starts with a default that reads many common session headers. Replace it with this expression:

    Text
    'session-id' in request.headers ? request.headers['session-id'][0] : ''
    The key returns the session-id header, or an empty string, which means no affinity, when the header is missing.
  4. In Priorities and weights, set Priority to 0 for both models, and keep both weights at 1. Share within priority shows 50% for each.

  5. In Routing order, keep Session affinity before priority.

    Session affinity before priority favors cache reuse. Priority before session affinity favors the primary priority group.
  6. Select Save. New requests use the new settings within a few seconds.

Checkpoint

The alias Backend models tab shows both models with Always included, at priority 0 and weight 1.

Watch the sessions

Send four requests for each of a few sessions, each with its own session-id header, plus four requests without the header (none). The script prints the backend model of each request, then stays when all of a session's requests went to one model, moves when they didn't, or inconclusive when a request failed.

# Send 4 requests for each session, with a session-id header, and 4 without one, and print each backend model
echo "📊 Backend model of each request, per session"
./infra/count-sessions.sh "$ALIAS" 4 alpha bravo charlie delta echo none
Text
📊 Backend model of each request, per session
Sending 4 requests per session with "model": "chat" and a session-id header
Session   Backend model of each request
alpha     gpt-6.1-sol  gpt-6.1-sol  gpt-6.1-sol  gpt-6.1-sol   stays
bravo     gpt-6.1-sol  gpt-6.1-sol  gpt-6.1-sol  gpt-6.1-sol   stays
charlie   gpt-5-mini   gpt-5-mini   gpt-5-mini   gpt-5-mini    stays
delta     gpt-6.1-sol  gpt-6.1-sol  gpt-6.1-sol  gpt-6.1-sol   stays
echo      gpt-6.1-sol  gpt-6.1-sol  gpt-6.1-sol  gpt-6.1-sol   stays
none      gpt-5-mini   gpt-6.1-sol  gpt-5-mini   gpt-6.1-sol   moves

Every session stays on one model, from its first request on, while the requests without a key alternate between the two models by weight. Which model a given session maps to depends on the key and on the alias's backend models, so your sessions can land differently. Run the script again: each session keeps its model.

Checkpoint

Every named session shows stays, and the sessions are spread over both models. The none row shows both models. An HTTP-429 entry, with inconclusive at the end of the row, means that a deployment reached its requests per minute, not that the session moved: wait a minute and run the script again.

Verify

1

Read the alias

# Read the alias, and print its routing
echo "🔖 Alias $ALIAS routes to:"
az aigateway model-alias show -g "$RG" --gateway-name "$GATEWAY" -n "$ALIAS" | jq -r "$ROUTES"
Checkpoint

Both backend models show priority 0 weight 1 always included, followed by Routing order: sessionBackendPreference, priority and the session-id key. On the portal track, the research model shows as gpt-6-1-sol, the name that the import gave it.

2

Check that sessions keep their model

# Send 3 requests for each of three sessions, and print each backend model
echo "📊 Backend model of each request, per session"
./infra/count-sessions.sh "$ALIAS" 3 alpha delta foxtrot
Checkpoint

Each session shows stays, and alpha and delta keep the models that they had in Watch the sessions.

Go further

  • Choose the routing order. Give gpt-6.1-sol priority 1. With Session affinity before priority (["sessionBackendPreference", "priority"]), sessions that map to gpt-6.1-sol stay there, which favors cache reuse. With Priority before session affinity (["priority", "sessionBackendPreference"]), every session goes to gpt-5-mini while it's available, and affinity only chooses among backend models that share the best priority. Use the second order when a lower priority means a backup, such as another region.
  • Combine a condition with affinity. Put the research condition back on gpt-6.1-sol at priority 0, with gpt-5-mini at priority 1, and keep the session key. With session affinity first, a research request in a session that maps to gpt-5-mini stays on gpt-5-mini. With priority first, research requests go to gpt-6.1-sol, and the other requests to gpt-5-mini.
  • Keep research requests off the general model. Priority still lets a research request fall back to gpt-5-mini when gpt-6.1-sol is unavailable. To prevent that, give gpt-5-mini the opposite condition, !('x-workload' in request.headers && request.headers['x-workload'][0] == 'research'). When every backend model is excluded, the gateway answers HTTP 503 without calling a provider.
  • Route by caller instead of by header. Create a runtime key named research-app, and use the condition has(authorization.key) && authorization.key.resource.name == 'research-app', so that only that key reaches the larger model. Requests that use a Microsoft Entra token can test claims instead, such as has(authorization.jwt) && 'roles' in authorization.jwt.claims && 'research' in authorization.jwt.claims['roles'].
  • Use the portal's default session key. It reads the first non-empty header among common session headers of agent harnesses and SDKs, such as x-ai-session-id, x-claude-code-session-id, thread-id, x-ms-session-id, session-id, and x-session-id, so most agents get affinity without changes.

Clean up

Keep the gateway, the runtime key, and gpt-5-mini from lab 02. Later labs use them. Remove what you added in this lab: the alias first, then the backend model, then the Foundry deployment.

Choose your track
1

Remove the alias and the research model

In ModelsAliases, select More actions on the chat row, then Delete and Delete alias. Requests that use the alias fail immediately; the backend models stay in the gateway. Then delete the gpt-6.1-sol model from ModelsBackend models. Finally, delete the gpt-6.1-sol deployment from your Foundry account, for example with the last command of the CLI track.

azd down and terraform destroy don't touch the gateway, key, and gpt-5-mini from lab 02, because the Bicep + azd and Terraform tracks created their own gateways.

Next steps

Finished the lab?

Mark it complete to track your progress across paths. It's saved only in this browser.