Azure AI Gateway Labs
Documentation Open portal
Security & identity ยท Lab 10

Issue and rotate runtime API keys

Give every app its own runtime API key, keep the value out of your code, rotate it without downtime with the primary and secondary values, revoke a leaked key, and see usage and cost per key in Monitoring.

Beginner 15 min PortalCLIBicepTerraform Updated
Source Edit

What you'll do

  • Create one runtime API key for each app and keep its value in a secret store, not in code
  • Declare the keys with Bicep and azd or Terraform, and keep rotation and revocation as operations
  • Rotate a key without downtime by moving clients between its primary and secondary values
  • Revoke a leaked key and confirm that the gateway rejects it with 401
  • Find usage and estimated cost for each key in Monitoring
  • Decide when to replace API keys with Microsoft Entra ID

Before you start

Tracks: Portal, Azure CLI, Bicep + azd, Terraform

Three callers now use your gateway: the app-dev web app, the support-bot agent, and a data scientist's notebook that was shared with the whole company, key included. In this lab you give each caller its own runtime API key, keep the values in a secret store, rotate the bot's key without taking it offline, and revoke the leaked notebook key. Because every key is a separate caller identity, you can also see who used what.

Before you begin

One-time setupThe Setup guide tasks that this lab uses. Mark each one done once, and the labs stop listing it.

Tip

On the Bicep + azd and Terraform tracks, azd up or terraform apply creates its own gateway, model, and keys, so you can skip this section. You need only the one-time setup above.

  • Complete Provision your AI Gateway. You need the gateway, its resource group, and its runtime URL.
  • Recommended: complete Publish a Foundry model and make your first call, so you have a model alias to call. The checks in this lab call that alias. The gateway answers 404 for an alias it doesn't know before it checks the key, so without a real alias the checks can't tell accepted keys from rejected keys.
  • Permission to manage the gateway's keys, for example Contributor on its resource group. Reading a key value needs Microsoft.ApiManagement/aigateways/apiKeys/listSecrets/action, which Contributor includes.
  • Optional: an Azure Key Vault that uses Azure RBAC, where you have Key Vault Secrets Officer. You can create one below.
  • Optional, for usage and cost per key: monitoring configured in Monitor usage, cost, and traces.
  • Run the scripts and samples from the lab's folder, ~/aigateway/labs/10-api-keys.

Set these variables once. Every track and the Verify section reuse them.

# Name your gateway, its resource group, and the model alias that the checks in this lab call
export RG="<resource-group>"
export GATEWAY="<gateway>"
export AI_GATEWAY_MODEL="<model-name>"      # an alias you can call, for example the one from lab 02
# export KEY_VAULT="<key-vault>"             # optional: uncomment to keep key values in Key Vault

# Read the gateway's runtime URL
export AI_GATEWAY_URL="https://$(az aigateway show -g "$RG" -n "$GATEWAY" --query properties.frontend.defaultHostname -o tsv)"
echo "๐Ÿ“ Gateway $GATEWAY in $RG, model alias $AI_GATEWAY_MODEL"
echo "๐ŸŒ Runtime URL: $AI_GATEWAY_URL"
No Key Vault yet? Create one for this lab.

Key Vault names are globally unique, 3 to 24 characters. The vault uses Azure RBAC, and you get Key Vault Secrets Officer so you can write and read secrets. A new role assignment can take a few minutes to apply.

# Name the vault with a random number, because Key Vault names are globally unique
export KEY_VAULT="kv-aigw-$RANDOM"

# Create the vault with Azure RBAC authorization, so that roles decide who can read its secrets
az keyvault create -g "$RG" -n "$KEY_VAULT" --enable-rbac-authorization true -o none &&
  echo "โœ… Key Vault is ready in $RG, and KEY_VAULT holds its name"

# Give yourself the Key Vault Secrets Officer role on the vault, so that you can write and read secrets
az role assignment create --role "Key Vault Secrets Officer" \
  --assignee "$(az ad signed-in-user show --query id -o tsv)" \
  --scope "$(az keyvault show -n "$KEY_VAULT" --query id -o tsv)" -o none &&
  echo "๐Ÿ” You're a Key Vault Secrets Officer on the vault (it can take a few minutes to apply)"

How it works

A runtime API key is a gateway resource with a name and two values, primary and secondary. Both values work at the same time, which is what makes rotation without downtime possible. Clients send either value as Authorization: Bearer <key>, which is how OpenAI SDKs send their API key, or in an api-key header. The gateway validates the key before it applies policies or forwards the request, so a missing, wrong, rotated, or deleted value, or a suspended key, gets 401 Unauthorized and never reaches a model. The 401 message is the same one that Microsoft Entra ID callers get: Unauthorized. Missing or invalid Entra ID token, or missing Data.All scope.

A key is gateway-scoped: it grants access to every published model and MCP server on the gateway. Each key is also a separate caller identity, which token limits, cost limits, and Monitoring use to tell callers apart. That's why every app, agent, or user should get its own key.

TaskPortal (AccessAPI keys)Azure CLI (az aigateway api-key)Bicep (aigateways/apiKeys)
Create a keyCreate API keycreateDeclare the key
Read a valueView keys (primary value)list-secrets (primary and secondary values)โ€” (never an output)
RotateRotate key (primary value)regenerate --key-type primary or --key-type secondaryโ€”
Block temporarilyโ€”update --state suspended, then --state activeโ€”
RevokeDeletedeleteโ€” (removing the key from the template doesn't delete it)

The gateway also has a built-in key named master. Don't hand it out; create named keys instead.

Create a key for each app

Create three keys: app-dev for the web app, support-bot for the agent, and old-notebook, which stands in for the key that leaked. Pick one track.

Choose your track
1

Create the first key

  1. In the AI Gateway portal, open your gateway and select AccessAPI keys.
  2. Select Create API key.
  3. In Key name, enter app-dev, and select Create.
  4. The API key created dialog shows the value. Select Copy & done.

Store the value right away; the next section shows where. After you close the dialog, only people with the apiKeys/listSecrets/action permission, such as Contributors, can display the value again, with View keys.

2

Create the other two keys

Repeat the steps for support-bot and old-notebook. Names must be unique, ignoring case; a duplicate gets A key named "..." already exists. The portal derives the key's resource name from Key name: lowercase, with every character other than letters, digits, and hyphens replaced by -. Use that resource name with the CLI.

3

Load a value in your shell

Select the โ€ฆ (More actions) button on the app-dev row, select View keys, and copy the value. Paste it into your shell for the checks in this lab:

# Paste the key without showing it, and export it for the checks in this lab
printf "AI Gateway API key: " && read -rs AI_GATEWAY_API_KEY && export AI_GATEWAY_API_KEY && echo
echo "๐Ÿ”‘ Key: in AI_GATEWAY_API_KEY (${#AI_GATEWAY_API_KEY} characters, not shown)"

From the lab folder, send one request with the app-dev key:

# Send one request with the app-dev key: the script prints the HTTP status, and only the key's last four characters
echo "๐Ÿ”‘ Expect 200 OK for the app-dev key:"
./infra/check-key.sh app-dev

The script sends a tiny Chat Completions request and prints only the last four characters of the key. On GitHub, open check-key.sh or check-key.ps1.

infra/check-key.shGitHub
#!/usr/bin/env bash
# Sends one small Chat Completions request through AI Gateway and reports whether the gateway accepted the
# runtime API key. Only the last four characters of the key are printed.
#
# Usage:
#   export AI_GATEWAY_URL="<gateway-url>"
#   export AI_GATEWAY_MODEL="<model-name>"
#   export AI_GATEWAY_API_KEY="..."                 # the key to test; never commit it
#   ./check-key.sh ["label"]                        # test AI_GATEWAY_API_KEY
#   AI_GATEWAY_API_KEY="$OLD_KEY" ./check-key.sh "old value"
#   ./check-key.sh --no-key                         # send the request without an Authorization header
# Requires curl.
set -euo pipefail

: "${AI_GATEWAY_URL:?Set AI_GATEWAY_URL to the gateway runtime URL, for example https://aigw-labs-k3v7qz2m9xw4p.eastus2-01.ai.gateway.azure.com}"
: "${AI_GATEWAY_MODEL:?Set AI_GATEWAY_MODEL to a callable model alias}"

URL="${AI_GATEWAY_URL%/}/default/models/openai/v1/chat/completions"
BODY=$(printf '{"model":"%s","messages":[{"role":"user","content":"ping"}]}' "$AI_GATEWAY_MODEL")

if [[ "${1:-}" == "--no-key" ]]; then
  LABEL="no key"
  HINT="-"
  RESPONSE=$(curl -sS -w '\n%{http_code}' "$URL" \
    -H "Content-Type: application/json" -d "$BODY" || true)
else
  : "${AI_GATEWAY_API_KEY:?Set AI_GATEWAY_API_KEY to the runtime key to test}"
  LABEL="${1:-current key}"
  HINT="...${AI_GATEWAY_API_KEY: -4}"
  # The header is read from stdin so the key doesn't appear in the process list.
  RESPONSE=$(printf 'Authorization: Bearer %s\n' "$AI_GATEWAY_API_KEY" | curl -sS -w '\n%{http_code}' "$URL" \
    -H @- -H "Content-Type: application/json" -d "$BODY" || true)
fi
STATUS="${RESPONSE##*$'\n'}"

case "$STATUS" in
  2??) RESULT="accepted" ;;
  401) RESULT="rejected: the key is missing, invalid, rotated, suspended, or deleted" ;;
  404) if [[ "$RESPONSE" == *model_not_found* ]]; then
         RESULT="not checked: the gateway doesn't know the alias in AI_GATEWAY_MODEL"
       else
         RESULT="key accepted, but the call failed: check AI_GATEWAY_MODEL and the gateway's policies"
       fi ;;
  000|"") RESULT="no response: check AI_GATEWAY_URL and your network"; STATUS="---" ;;
  *)   RESULT="key accepted, but the call failed: check AI_GATEWAY_MODEL and the gateway's policies" ;;
esac

printf '%-14s %-8s HTTP %s  %s\n' "$LABEL" "$HINT" "$STATUS" "$RESULT"
Checkpoint

AccessAPI keys lists app-dev, support-bot, and old-notebook, and the check prints HTTP 200 accepted for app-dev.

Send the key in either header

The gateway reads a runtime key from one of two headers. Use one per request; a request with both gets 400.

HeaderUsed by
Authorization: Bearer <key>The docs, OpenAI SDKs, and the checks in this lab
api-key: <key>Azure OpenAI SDKs and tools that send an Azure OpenAI key

Send the same request with each header, and then with both:

# Send the same request once with each header, and print only the HTTP status
URL="${AI_GATEWAY_URL%/}/default/models/openai/v1/chat/completions"
BODY="$(printf '{"model":"%s","messages":[{"role":"user","content":"ping"}]}' "$AI_GATEWAY_MODEL")"
for HEADER in "Authorization: Bearer" "api-key:"; do
  # Read the header from stdin, so that the key doesn't appear in the process list
  STATUS="$(printf '%s %s\n' "$HEADER" "$AI_GATEWAY_API_KEY" | curl -sS -o /dev/null -w '%{http_code}' "$URL" \
    -H @- -H "Content-Type: application/json" -d "$BODY")"
  echo "๐Ÿ”‘ ${HEADER%%:*} header: HTTP $STATUS"
done

# Send both headers in one request: the gateway rejects it with 400
echo "๐Ÿšฆ Expect 400 Bad Request for both headers:"
printf 'Authorization: Bearer %s\napi-key: %s\n' "$AI_GATEWAY_API_KEY" "$AI_GATEWAY_API_KEY" |
  curl -sS "$URL" -H @- -H "Content-Type: application/json" -d "$BODY" -w '\nHTTP %{http_code}\n'
Checkpoint

Both headers print HTTP 200, and the request with both prints HTTP 400 with "Specify either an API key or an Authorization header, not both."

Store the key outside your code

A runtime key opens every model and MCP server on the gateway, so treat it like a password:

  • Local development: set an environment variable for the session, as in this lab, and read it in code, for example os.environ["AI_GATEWAY_API_KEY"] in Python. If you keep it in a .env file, add the file to .gitignore.
  • Hosted apps: keep the value in Azure Key Vault. App Service and Azure Functions resolve Key Vault references in app settings, and Azure Container Apps can reference Key Vault secrets.
  • Pipelines: use the secret store of your CI system, or read the value from Key Vault at run time.
  • Never put the value in source code, a committed .env file, build logs, notebooks, chat, or tickets.

If you set KEY_VAULT, store the app-dev and support-bot values there. The commands print only the secret IDs. Each secret is named <key>-api-key, which is the name the rotation script updates later.

# Copy the primary value of each key into a Key Vault secret named <key>-api-key, and print only the secret IDs
echo "๐Ÿ†” Key Vault secret IDs:"
for KEY in app-dev support-bot; do
  az keyvault secret set --vault-name "$KEY_VAULT" -n "$KEY-api-key" \
    --value "$(az aigateway api-key list-secrets -g "$RG" --gateway-name "$GATEWAY" -n "$KEY" --query primaryKey -o tsv)" \
    --tags aigateway-key="$KEY" key-type=primary --query id -o tsv
done

An app, or you, then reads the value from the vault instead of from the gateway:

# Read the app-dev value from Key Vault instead of from the gateway, without showing it
export AI_GATEWAY_API_KEY="$(az keyvault secret show --vault-name "$KEY_VAULT" -n app-dev-api-key --query value -o tsv)"
echo "๐Ÿ”‘ Key: in AI_GATEWAY_API_KEY (${#AI_GATEWAY_API_KEY} characters, not shown)"
Note

The portal track can store the values the same way: run the commands above, or add the secrets to the vault in the Azure portal.

Rotate a key without downtime

Rotate keys on a schedule, and whenever someone who had a value leaves the team. Regenerating a value invalidates that value only, and right away. The trick is to regenerate the value that no client is using.

On the Bicep + azd and Terraform tracks, follow the Azure CLI track. A later azd provision or terraform apply keeps the regenerated values.

Choose your track
1

Save the current value

You use the old value later to prove that it stopped working. On the support-bot row, select โ€ฆ (More actions) โ€บ View keys, copy the value, and paste it into your shell:

# Paste the current support-bot value without showing it, to prove later that the rotation retired it
printf "Current support-bot value: "; read -rs OLD_SUPPORT_BOT; echo
echo "๐Ÿ”‘ Current support-bot value: in OLD_SUPPORT_BOT (${#OLD_SUPPORT_BOT} characters, not shown)"
2

Rotate the key

  1. On the support-bot row, select โ€ฆ (More actions) โ€บ Rotate key.
  2. The Rotate "support-bot" key? dialog warns that clients using the old value get 401 Unauthorized until you update them. Select Rotate.
  3. In API key rotated, copy the new value and update the client, or the support-bot-api-key secret in Key Vault.
Note

The portal rotates the primary value in place, so a client that still sends the old value gets 401 until you update it. To rotate without downtime from the portal, replace the key instead: create support-bot-v2, move the client to it, and then delete support-bot. The new key is a new caller identity, so move any cost-limit overrides to it, and expect Monitoring to report it separately. The CLI track avoids both trade-offs with the secondary value.

Checkpoint

AI_GATEWAY_API_KEY="$OLD_SUPPORT_BOT" ./infra/check-key.sh "rotated value" (or ./infra/check-key.ps1 -ApiKey $OLD_SUPPORT_BOT -Label "rotated value") prints HTTP 401.

Revoke a leaked key

When a key leaks, revoke it first and investigate afterward. Deleting a key invalidates both of its values within about 10 seconds, and anyone who holds them then gets 401. Pick one track.

On the Bicep + azd track, follow the Azure CLI track. Then remove old-notebook from the template's list, and keep any names you added, so that a later azd provision doesn't create it again, with new values:

# Keep only the keys you still need in the list, so that a later azd provision doesn't create old-notebook again
azd env set AI_GATEWAY_API_KEYS '["app-dev","support-bot"]' &&
  echo "๐Ÿ“ AI_GATEWAY_API_KEYS lists app-dev and support-bot"

On the Terraform track, follow the Azure CLI track too, then come back and run this block. If you run it first, Terraform deletes old-notebook before the CLI track saves its value for Verify. Plan without old-notebook, and keep any names you added: a plan with the default list creates old-notebook again, with new values (Plan: 1 to add, 0 to change, 0 to destroy.). Pass the same list to every later plan, or change the default in terraform/variables.tf:

# After the CLI deletes old-notebook, plan with only the keys you still need: the plan changes only an output
cd ~/aigateway/labs/10-api-keys/terraform &&
  terraform plan -var 'api_key_names=["app-dev","support-bot"]' -out tfplan &&
  terraform apply tfplan &&
  echo "๐Ÿ“ Terraform manages app-dev and support-bot"
cd ..
Choose your track
1

Save the leaked value

You use it to prove that the key no longer works. On the old-notebook row, select โ€ฆ (More actions) โ€บ View keys, copy the value, and paste it into your shell:

# Paste the leaked value without showing it, to prove later that the gateway rejects it
printf "Leaked value: "; read -rs LEAKED_KEY; echo
echo "๐Ÿ”‘ Leaked value: in LEAKED_KEY (${#LEAKED_KEY} characters, not shown)"
2

Delete the key

Select the check box on the old-notebook row, and then select Delete. The Delete "old-notebook" key? dialog warns that applications using the key will no longer authenticate to the gateway and that the action can't be undone. Select Delete.

Then clean up after the leak: remove the value from the notebook and its history, and check Monitoring for calls made with the key.

Note

If the caller behind a leaked key must keep its identity, for example because a cost-limit override or Monitoring history is tied to it, regenerate both values instead of deleting the key. Run ./infra/rotate-key.sh <key> primary and then ./infra/rotate-key.sh <key> secondary (or the PowerShell script with -KeyType), and give the caller the new value.

See usage and cost per key

Each key is a caller identity, so Monitoring can break usage down by key. This needs the monitoring setup from Monitor usage, cost, and traces and a few requests with each key.

  1. Select MonitoringModels.
  2. Active caller identities counts the API keys and Microsoft Entra identities with observed activity.
  3. Model usage by caller identity compares caller identities by estimated model cost, tokens, and calls.
  4. Use the All caller identities filter to scope the page to one key, for example support-bot. Keys appear by name, marked API key.

The same identity drives enforcement. A Cost limit counted by Caller identity gives each key its own budget:

A Cost limit counted by caller identity: each API key gets its own $5 daily budget, and a caller that reaches it gets 403 LlmCostQuotaExceeded.

To set budgets and limits per key, continue with Enforce cost budgets with per-caller overrides.

When to move to Microsoft Entra ID

API keys are the fastest way to start, but the docs don't recommend them for production.

Runtime API keyMicrosoft Entra ID with Azure RBAC
What the gateway seesOne named keyA user, agent, or workload identity in your tenant
What it can callEvery published model and MCP serverWhat its role grants: AI Gateway User (models and tools), AI Gateway Models User, or AI Gateway Tools User, on all assets, one alias, or one MCP server
Secret to protectThe key valueNone with a managed identity; tokens are short-lived
RotationYou rotate and redistribute valuesMicrosoft Entra ID issues new tokens
Good fitPrototypes, labs, and tools that only accept a keyProduction apps, agents, and per-user access

Move to Microsoft Entra ID with Go keyless with Microsoft Entra ID and RBAC.

Verify

Run the checks from the lab folder. Only current values should get through; the rotated value, the deleted key, and a request without a key should all get 401.

1

Load the app-dev key

If AI_GATEWAY_API_KEY no longer holds the app-dev value, load it again from Key Vault, with az aigateway api-key list-secrets, or by pasting it, as in Create a key for each app.

2

Run the checks

# Check the current app-dev key, which the gateway accepts
echo "๐Ÿ”‘ Expect 200 OK for the app-dev key:"
./infra/check-key.sh app-dev

# Check the rotated value, the deleted key, and a request without a key: the gateway rejects all three
echo "๐Ÿšฆ Expect 401 Unauthorized for the rotated value, the deleted key, and no key:"
AI_GATEWAY_API_KEY="$OLD_SUPPORT_BOT" ./infra/check-key.sh "rotated value"
AI_GATEWAY_API_KEY="$LEAKED_KEY" ./infra/check-key.sh "deleted key"
./infra/check-key.sh --no-key

Expected output (your key endings differ):

Text
๐Ÿ”‘ Expect 200 OK for the app-dev key:
app-dev        ...Q7xk  HTTP 200  accepted
๐Ÿšฆ Expect 401 Unauthorized for the rotated value, the deleted key, and no key:
rotated value  ...m2Rd  HTTP 401  rejected: the key is missing, invalid, rotated, suspended, or deleted
deleted key    ...Vt9a  HTTP 401  rejected: the key is missing, invalid, rotated, suspended, or deleted
no key         -        HTTP 401  rejected: the key is missing, invalid, rotated, suspended, or deleted
3

Check that the rotated key still works

Load a current support-bot value from View keys, Key Vault, or az aigateway api-key list-secrets, and run the check again. After the CLI flow, both the new primary and the new secondary value work. The check prints HTTP 200:

# Paste a current support-bot value without showing it
printf "Current support-bot value: "; read -rs SUPPORT_BOT; echo
echo "๐Ÿ”‘ Current support-bot value: in SUPPORT_BOT (${#SUPPORT_BOT} characters, not shown)"

# Check it: after the rotation, the gateway accepts both the new primary and the new secondary value
echo "๐Ÿ”‘ Expect 200 OK for the current support-bot value:"
AI_GATEWAY_API_KEY="$SUPPORT_BOT" ./infra/check-key.sh support-bot

Go further

  • Prompt for the key in MCP clients. VS Code can ask for the key when it starts an MCP server instead of storing it in mcp.json. See Federate MCP tools behind one endpoint.
  • Get a rotation reminder. Set an expiry date on the Key Vault secret with az keyvault secret set-attributes --expires, and subscribe to the vault's SecretNearExpiry event.
  • Automate rotation. Run the rotation script from a scheduled pipeline that signs in with a workload identity, with a pause between the secondary and primary steps for clients to reload.
  • Limit each key. Give every key its own token allowance with Throttle tokens per caller with token limits.

Clean up

Keep app-dev and support-bot: Enforce cost budgets with per-caller overrides gives them different budgets.

Choose your track
1

Delete the test keys

Delete any other keys you created for testing, such as support-bot-v2, in AccessAPI keys (check box, then Delete).

If you created a Key Vault only for this lab, delete it. Key Vault keeps deleted vaults for a retention period and reserves the name; purge it if you want to reuse the name right away:

Bash
# Delete the vault: Key Vault keeps it for a retention period, and reserves its name
az keyvault delete -n "$KEY_VAULT" &&
  echo "๐Ÿšฎ Key Vault is deleted, and its name stays reserved until you purge it"

# Purge the deleted vault, so that you can reuse its name right away
az keyvault purge -n "$KEY_VAULT" &&
  echo "๐Ÿšฎ Key Vault is purged"

Clear the key values from your shell with unset AI_GATEWAY_API_KEY OLD_SUPPORT_BOT LEAKED_KEY SUPPORT_BOT (Bash) or Remove-Variable OLD_SUPPORT_BOT, LEAKED_KEY, SUPPORT_BOT; Remove-Item Env:AI_GATEWAY_API_KEY (PowerShell). Delete the gateway from earlier labs only when you finish all labs, as described in Provision your AI Gateway. azd down and terraform destroy don't touch it, because the Bicep + azd and Terraform tracks created their own gateway.

Next steps

Finished the lab?

Mark it complete to track your progress across paths. It's saved only in this browser.