Azure AI Gateway Labs
Documentation Open portal
Governance & cost · Lab 07

Enforce cost budgets with per-caller overrides

Give every app, agent, and user a dollar budget per hour, day, or month, raise or lower it for specific API keys and Microsoft Entra identities, and watch the gateway answer 403 LlmCostQuotaExceeded once a caller's estimated spend runs out.

Intermediate 30 min PortalCLIBicepTerraformNotebook Updated
Notebook Source Edit

What you'll do

  • Explain how a cost limit estimates spend and counts it per caller in fixed calendar windows
  • Add a daily cost limit to a model in the portal or with the Azure CLI
  • Give specific API keys and Microsoft Entra identities their own budget with overrides
  • Declare a budgeted alias, its API keys, and their overrides with Bicep and azd or with Terraform
  • Trigger a 403 LlmCostQuotaExceeded response, find the remaining budget header, and review estimated spend in Monitoring

Before you start

Tracks: Portal, Azure CLI, Bicep + azd, Terraform, Notebook

Your FinOps team has a problem that token limits can't solve: a thousand tokens on a premium reasoning model costs many times more than a thousand tokens on a mini model, so "tokens per day" says nothing about the bill. They want every app to have a daily budget in dollars, a bigger envelope for the production support bot, and a separate allowance for a data science lead who signs in with Microsoft Entra ID. In this lab you set that up as a cost limit at the gateway, so a caller who runs out of budget is stopped before the model is called.

Before you begin

One-time setupThe Setup guide tasks that this lab uses. Mark each one done once, and the labs stop listing it.

Tip

On the Bicep + azd and Terraform tracks, azd up or terraform apply creates its own gateway, model, and keys, so you can skip this section. You need only the one-time setup above.

  • Complete Throttle tokens per caller with token limits. This lab reuses the PoliciesAdd policy wizard and the idea of per-caller counters.
  • Complete Issue and rotate runtime API keys. You need two runtime keys from AccessAPI keys, for example app-dev (gets the default budget) and support-bot (gets an override).
  • Recommended: configure Application Insights in Monitor usage, cost, and traces and send a few requests first, so Monitoring can show estimated spend before you choose a budget.
  • Optional, for the Entra override: a user or service principal that can call the gateway with Microsoft Entra ID. Go keyless with Microsoft Entra ID and RBAC shows how.
  • Permission to change the gateway, for example Contributor on the resource group.
  • Run the scripts and samples from the lab's folder, ~/aigateway/labs/07-cost-budgets.

Set these variables once. Both tracks and the Verify section reuse them. Use the app-dev key here.

# Name your gateway, its model provider, and the backend model that gets the budget
export RG="<resource-group>"
export GATEWAY="<gateway>"                 # 24 characters or fewer
export PROVIDER="<provider-name>"          # for example foundry-models
export MODEL="<model-name>"                # the backend model name under that provider

# Read the gateway's resource ID, which the key commands use
export GATEWAY_ID="$(az aigateway show -g "$RG" -n "$GATEWAY" --query id -o tsv)"
echo "🆔 Gateway: $GATEWAY_ID"

# Read the gateway's runtime URL, which apps call
export AI_GATEWAY_URL="https://$(az aigateway show -g "$RG" -n "$GATEWAY" --query properties.frontend.defaultHostname -o tsv)"
echo "🌐 Runtime URL: $AI_GATEWAY_URL"

# Read the alias that apps send in the model field: for Foundry imports, it matches the deployment's model name
export AI_GATEWAY_MODEL="$(az aigateway model show -g "$RG" --gateway-name "$GATEWAY" --provider-name "$PROVIDER" \
  -n "$MODEL" --query properties.deployment.modelName -o tsv)"
echo "🔖 Model alias: $AI_GATEWAY_MODEL"

# Paste the app-dev runtime key from Access › API keys, without showing it: don't commit it or write it to a file
printf "AI Gateway API key: " && read -rs AI_GATEWAY_API_KEY && export AI_GATEWAY_API_KEY && echo
echo "🔑 Key: in AI_GATEWAY_API_KEY (${#AI_GATEWAY_API_KEY} characters, not shown)"

How it works

After each model response, the gateway estimates its cost from the tokens the provider reported and the model's public pricing, broken down by token category. A cost limit adds that estimate to the caller's counter for the current renewal period. Before it forwards the next request, the gateway checks the counter. Once spend reaches the budget, the gateway answers 403 (LlmCostQuotaExceeded in the product docs) with the message LLM cost quota is exceeded and never calls the backend. Answered responses carry a header with the estimated cost of the call, and blocked responses carry the remaining budget and the seconds until it renews.

SettingPortal labelJSON fieldValues
BudgetBudget amountamountUS dollars, for example 5
WindowRenewal periodperiodhour, day, week, month, year (Hourly … Yearly)
Who gets a counterCounter dimensions (shown as Budget counted by)counterKeyA required array: ["identity"] (the API key, or the Entra tenant ID + object ID), ["ipAddress"], or [] for one counter that all callers share
ExceptionsOverridesoverridesA different amount for an API key (apiKeyResourceId) or Entra identity (claims: a list of name and value pairs that must all match)
Policy identityNot shownidRequired, and unique on the asset. Editing the amount keeps accumulated spend.

Renewal periods are fixed calendar windows, not rolling windows: a Daily budget starts fresh at the beginning of each day, not 24 hours after the first call. Attach the policy to a model to budget spend on that model, or at Gateway scope to track one budget per caller across all models.

Important

A cost limit is a guardrail, not an invoice. Spend is estimated from public pricing, and because gateway instances update counters concurrently, concurrent requests can briefly overshoot the budget before every instance sees the latest spend.

Add a cost limit to the model

Pick one track. Each one produces the same default budget: $5 per day for each caller identity. The portal and Azure CLI tracks put it on your model. The Bicep + azd and Terraform tracks create a new gateway and put it on a new alias of the model. Every track also adds the overrides for specific callers.

Choose your track
1

Start the policy wizard

Open the AI Gateway portal and select your gateway from the gateway switcher. Select PoliciesAdd policy. On the Type step, under Cost and rate limits, select Cost limit, then Next.

2

Choose where the budget applies

On the Scope step, select Individual assets ("Apply only to selected models, aliases, or MCP servers"). Select the row of type Model for your model, then Next.

Tip

Choose Gateway ("Apply to the entire gateway workspace, including applicable assets added later") when the budget should cover every model a caller uses, including models you import later. A gateway-scoped cost limit tracks one budget across all models instead of one per model.

3

Set the default budget

On the Details step, under Parameters, set:

FieldValue
Budget amount5 (US dollars)
Renewal periodDaily
Counter dimensionsIdentity

Select Next.

Note

Identity gives each API key, and each Entra caller, its own counter. IP address counts per client IP address instead, and can't be combined with overrides: while the policy has overrides, Edit policy disables Counter dimensions and shows "Remove all overrides before switching to Caller IP address."

4

Give an API key a bigger budget

On the Overrides step, under API keys, select Add. In the new row, pick your support-bot key in API key and set Budget amount to 50. Each key can have only one override.

5

Give a Microsoft Entra identity its own budget

Under Microsoft Entra identities, select Add. In Add Microsoft Entra identities override, choose how Match requests by works:

  • Specific Microsoft Entra identity: search for a user by name or email. For a service principal, enter its object ID, not its application (client) ID.
  • Custom token claim combination: add one or more Claim name and Value pairs, for example appid and the client ID of an internal app. Every claim condition must match the validated token.

Set Budget amount to 20, then select Add.

Overrides (optional): API key overrides match a specific key; Microsoft Entra overrides match one identity or a combination of validated token claims. The first matching override wins.

API key and Microsoft Entra overrides are evaluated together, in the order you add them. The first match wins, and later overrides aren't evaluated. A claim combination never pools callers into a shared budget: with identity counting, each matching identity gets the override amount on its own counter.

6

Create the policy

Select Create. The portal confirms Policy created, and Policies lists the Cost limit for your model with $5.00 / day per caller identity under Configuration. Select it to review the policy.

The details page explains the trigger and the 403 LlmCostQuotaExceeded response, and lists the Policy ID, Budget amount, Renewal period, Budget counted by, Overrides, and Applies to.
Checkpoint

The Cost limit details page shows Budget amount $5.00, Renewal period Daily, and Budget counted by Caller identity. Under Overrides, support-bot has $50.00, and your Microsoft Entra override shows as Claim-based override (read-only) with $20.00. Your model is under Applies to.

Verify

A $5 budget takes a while to spend, so temporarily lower it, run a short series of requests with the app-dev key (which has no override), and watch the gateway switch from 200 to 403.

1

Lower the budget for the test

A call to gpt-5-mini costs about $0.0004, so a budget of 0.001 is spent after about three calls. For a more expensive model, use 0.01. In the portal, open the Cost limit, select Edit, set Budget amount to 0.001, and Save. On the Azure CLI track, run Set the cost limit again with --amount 0.001. The new budget reaches the gateway within about a minute, so wait before you run the requests.

Because editing keeps accumulated spend, the very first request might already be blocked if app-dev spent more than the new budget earlier today.

On the Bicep + azd track, set the budget to 0.001 too:

# Lower the default budget to $0.001 for each caller, in the azd environment
azd env set COST_LIMIT_AMOUNT 0.001 &&
  echo "📝 COST_LIMIT_AMOUNT is 0.001"

# Deploy the change: only the budgeted alias changes, and it keeps its policy id and the spend so far
azd provision &&
  echo "🔐 Default budget on $AI_GATEWAY_MODEL is now \$0.001 for each caller"

On the Terraform track, in the lab folder, run terraform -chdir=terraform plan -var budget_amount=0.001 -out tfplan, then terraform -chdir=terraform apply tfplan. The plan ends with Plan: 0 to add, 1 to change, 0 to destroy., for azapi_resource.budget_alias, which keeps its policy id and the spend so far.

2

Run the requests

From the lab folder, run the script for your shell. It prints the status, the error code, the tokens used, and any response header whose name mentions budget, cost, or remaining.

# Send 10 requests with the app-dev key, and print the status, the tokens used, and the budget headers of each
echo "🚦 Expect 403 once app-dev's estimated spend reaches the budget:"
./infra/trigger-cost-limit.sh "$AI_GATEWAY_MODEL" 10

On GitHub, open trigger-cost-limit.sh or trigger-cost-limit.ps1.

Example output with a $0.001 budget on the alias from the Bicep + azd track (your token counts, the number of successful calls, and the header values will differ):

Text
🚦 Expect 403 once app-dev's estimated spend reaches the budget:
Request  Status  Error code               Tokens used  Budget headers
1        200     -                        205          …; x-alias-cost-per-day-consumed=0.000368000
2        200     -                        189          …; x-alias-cost-per-day-consumed=0.000336000
3        200     -                        196          …; x-alias-cost-per-day-consumed=0.000350000
4        200     -                        211          …; x-alias-cost-per-day-consumed=0.000380000
5        403     -                        -            x-alias-cost-per-day-retry-after=74726; x-alias-cost-per-day-consumed=0.000000000
…
10       403     -                        -            x-alias-cost-per-day-retry-after=74722; x-alias-cost-per-day-consumed=0.000000000

x-alias-cost-per-day-consumed is the estimated cost of an answered call; the elided headers are the Foundry deployment's x-ratelimit-* headers. A blocked call carries …-retry-after, the seconds until the budget renews at the start of the next day in UTC, and can also carry x-alias-cost-per-day-remaining, which is 0. On a cost limit attached to the model, the headers start with x-cost- instead. A blocked call has no error code in its body, so the script prints -. How many calls succeed depends on the model's price. A call or two more than you expect is the small overshoot described in How it works. If all 10 requests return 200, run the script again: the spend so far still counts.

3

Inspect one blocked response

Look at the full headers and body of a single blocked call:

Bash
# Send one more request with the app-dev key, and show the whole response: the status line, the headers, and the body
echo "🚦 Expect 403 with the message LLM cost quota is exceeded:"
curl -i "${AI_GATEWAY_URL%/}/default/models/openai/v1/chat/completions" \
  -H "Authorization: Bearer $AI_GATEWAY_API_KEY" -H "Content-Type: application/json" \
  -d "{\"model\":\"$AI_GATEWAY_MODEL\",\"messages\":[{\"role\":\"user\",\"content\":\"ping\"}]}"

The status line is HTTP/2 403, and the body is { "statusCode": 403, "message": "LLM cost quota is exceeded. Try again in 30 minutes and 36 seconds." }, with your own wait time. Scan the headers for the ones that report the cost and the remaining budget, and compare them with a successful response from a key that still has budget. Unlike a token limit, which frees up as the window rolls, the caller stays blocked until the next calendar renewal period starts or you raise the budget.

4

Prove the override and the per-caller counters

Set AI_GATEWAY_API_KEY to the support-bot key in a new terminal (repeat the variable block) and run the script again. Every request succeeds, because that key has its own counter and a $50 override. On the Bicep + azd and Terraform tracks, read that key from the gateway instead:

# Read the support-bot key from the gateway, so that it never shows on screen
export AI_GATEWAY_API_KEY="$(az aigateway api-key list-secrets --ids "$GATEWAY_ID/apiKeys/support-bot" \
  --query primaryKey --output tsv)"
echo "🔑 support-bot key: in AI_GATEWAY_API_KEY (${#AI_GATEWAY_API_KEY} characters, not shown)"

# Send the same 10 requests with support-bot, which has its own counter and a $50 override
echo "📊 Expect 200 for every request, because support-bot has its own \$50 budget:"
./infra/trigger-cost-limit.sh "$AI_GATEWAY_MODEL" 10

If you added an Entra override (the Bicep + azd and Terraform tracks add one for you), call the gateway with that identity's token as shown in Go keyless with Microsoft Entra ID and RBAC: it gets its own counter too, keyed by tenant ID and object ID.

5

Review estimated spend in Monitoring

If you configured Application Insights, open Monitoring. Estimated model cost and Top models show the spend the gateway estimated, and the dashboard attributes it to individual API keys and models.

Monitoring estimates model cost from observed tokens and public pricing, and breaks it down by model and API key.
Checkpoint

app-dev got 200 responses until its estimated spend reached the budget, then 403 with LLM cost quota is exceeded. support-bot wasn't affected, and the backend received no calls from app-dev while it was blocked.

Go further

  • Cap spend across all models. Add a second Cost limit with Gateway scope, for example $200 Monthly. Each caller then has one budget across every model, including models you import later. With the Azure CLI, run az aigateway policy set-cost-limit -g "$RG" --gateway-name "$GATEWAY" --amount 200 --period month --counter-key identity.
  • Combine with token limits. Keep a per-minute Token rate limit to smooth bursts and a Cost limit to cap spend. Both are checked on every request, and either one can block.
  • Budget by role or app. Use Custom token claim combination overrides, for example a roles claim with an app role value, so everyone with that role gets a different per-identity amount.
  • Chargeback by team. Export tid, oid, or a custom claim to cost metrics in Attribute tokens and cost with OpenTelemetry.

Clean up

A $0.001 budget blocks real traffic. Delete the policy, or raise it to a production value, before you continue. Raising the amount takes effect immediately, because the accumulated spend is kept and compared with the new budget.

Choose your track
1

Remove or raise the budget

Select Policies and open the Cost limit you created. Select More actions (⋮) › Delete, then Delete again in Delete this policy?, or select Edit, set a production Budget amount, and Save. If you created runtime keys only for this lab, revoke them in AccessAPI keys.

Keep the gateway, model, and keys from earlier labs: the next labs reuse them. azd down and terraform destroy don't touch them, because the Bicep + azd and Terraform tracks created their own gateways.

Next steps

Finished the lab?

Mark it complete to track your progress across paths. It's saved only in this browser.