Azure AI Gateway Labs
Documentation Open portal
Security & identity ยท Lab 12

Call backends with managed identities

Let the gateway reach Microsoft Foundry with a managed identity instead of stored keys. Check the system-assigned identity, then create a user-assigned identity you can share across gateways, grant it Foundry User, attach it, and point a model provider at it from the portal, the Azure CLI, Bicep, or Terraform.

Intermediate 20 min PortalCLIBicepTerraform Updated
Source Edit

What you'll do

  • Explain how the gateway authenticates to backends, separately from how callers authenticate to the gateway
  • Check the gateway's system-assigned identity and the assets that use it
  • Create a user-assigned identity, attach it to the gateway, and grant it Foundry User on a Foundry account
  • Point a Foundry model provider at the user-assigned identity and prove that calls still succeed
  • Decide when a backend needs a stored API key instead

Before you start

Tracks: Portal, Azure CLI, Bicep + azd, Terraform

Your platform team runs one gateway per region, and every gateway calls the same Foundry account. Today each gateway uses its own system-assigned identity, so every new gateway needs a fresh Foundry User grant, and recreating a gateway leaves a stale role assignment behind. The team wants one identity that it owns, grants once, and attaches to every gateway, with no backend key stored anywhere. In this lab you check the gateway's system-assigned identity, create a user-assigned identity, give it access to Foundry, and switch a model provider to it without changing a single client.

Before you begin

One-time setupThe Setup guide tasks that this lab uses. Mark each one done once, and the labs stop listing it.

Tip

On the Bicep + azd and Terraform tracks, azd up or terraform apply creates its own gateway, model, and key, so you can skip this section. You need only the one-time setup above, and permission to create role assignments, for example Owner on the subscription. To check the system-assigned identity first, run the track in Create a user-assigned identity and give it access up to Read the outputs and the key, which sets the variables, then come back.

  • Complete Publish a Foundry model and make your first call. You need a gateway with a Foundry model provider that uses a managed identity, a model alias, and a runtime API key.
  • Every track uses the Azure CLI at least once: the portal can't change the identity of a model provider that already exists.
  • Permissions:
    • Create a managed identity in the resource group, for example Contributor or Managed Identity Contributor.
    • Attach the identity to the gateway: Contributor on the gateway and Managed Identity Operator on the identity. Contributor on a resource group that holds both covers this.
    • Create role assignments on the Foundry account, for example Owner, User Access Administrator, or Role Based Access Control Administrator.

Managed identities are free. Nothing in this lab adds charges beyond the model calls you make.

Set these variables once. Every track and the Verify section reuse them.

# Set your gateway, its Foundry provider and Foundry account, and the name of the user-assigned identity to create
export RG="<resource-group>"
export GATEWAY="<gateway>"
export PROVIDER="<provider-name>"                 # the Foundry provider from lab 02, for example foundry-models
export FOUNDRY_RG="<foundry-resource-group>"      # resource group of the Foundry account
export FOUNDRY_ACCOUNT="<foundry-account-name>"   # the account name, not a project or deployment name
export IDENTITY="id-aigateway-backends"           # the user-assigned identity to create
echo "๐Ÿ“ Gateway $GATEWAY in $RG, with provider $PROVIDER and Foundry account $FOUNDRY_ACCOUNT"
echo "๐Ÿ‘ค Identity to create: $IDENTITY"

# Read the Foundry account's resource ID and the gateway's runtime URL, and set the alias that you call
export FOUNDRY_ID="$(az cognitiveservices account show -g "$FOUNDRY_RG" -n "$FOUNDRY_ACCOUNT" --query id -o tsv)"
export AI_GATEWAY_URL="https://$(az aigateway show -g "$RG" -n "$GATEWAY" --query properties.frontend.defaultHostname -o tsv)"
export AI_GATEWAY_MODEL="<model-name>"            # the alias you called in lab 02
echo "๐Ÿ†” Foundry account: $FOUNDRY_ID"
echo "๐ŸŒ Runtime URL: $AI_GATEWAY_URL"
echo "๐Ÿ”– Model alias: $AI_GATEWAY_MODEL"

# Paste the runtime key from lab 02 without showing it, and don't commit it or write it to a file
printf "AI Gateway API key: " && read -rs AI_GATEWAY_API_KEY && export AI_GATEWAY_API_KEY && echo
echo "๐Ÿ”‘ Key: in AI_GATEWAY_API_KEY (${#AI_GATEWAY_API_KEY} characters, not shown)"

Don't remember the provider name? List the Foundry providers: az aigateway model-provider list -g "$RG" --gateway-name "$GATEWAY" --query "[?properties.modelProviderType=='Foundry'].name" -o tsv.

How it works

Every call crosses two boundaries. Inbound, the caller proves who it is to the gateway with a runtime API key (lab 10) or a Microsoft Entra token (lab 11). Outbound, the gateway proves who it is to the backend. The caller never sends a backend credential: you configure it once on the asset, such as a model provider or an MCP server backend.

For Azure backends, use a managed identity. The gateway requests a Microsoft Entra token for the backend's audience, https://cognitiveservices.azure.com/ for Foundry, and Azure RBAC on the backend decides what the identity can do. For Foundry and Azure OpenAI, grant Foundry User on the account. Keep backend API keys for backends that can't accept an Entra token.

System-assigned identityUser-assigned identityStored backend key
Created byAzure, with the gateway. The portal enables it by default.You, as its own Azure resourceThe backend's owner
LifecycleTied to the gateway. Deleting the gateway deletes it.Independent. Attach it to many gateways and keep it when a gateway is recreated.You store it in the gateway and rotate it
Best forOne gateway calling Azure backendsSharing one identity and its role assignments across gatewaysThird-party providers and tools without Entra support
Foundry provider settingauthentication.managedIdentity without clientIdauthentication.managedIdentity.clientId set to the identity's client IDauthentication.headers, for example an api-key header

Attaching an identity to the gateway doesn't change which identity an asset uses. Each asset names its identity, so you can move assets one at a time.

The table uses the field names of API version 2026-11-01, where authentication sits at the top level of the provider's properties, next to foundry. The az aigateway output shows the same shape. Only its --foundry argument still takes authentication as one of the Foundry settings, and the CLI writes it to the top level.

Check the system-assigned identity

Lab 02's provider calls Foundry with the gateway's system-assigned identity. See it before you change anything.

1

Look at the identity in the portal

  1. Open the AI Gateway portal and select your gateway.
  2. Select Managed identities. On Configure identities, the System-assigned identity card shows Enabled with a Principal ID and a Tenant ID.
  3. Select View access. Backend access lists the assets that authenticate with a managed identity. Your Foundry provider appears with System-assigned as its identity and Foundry User as its role.
Configure identities shows the system-assigned identity's principal and tenant IDs, and the user-assigned identities attached to the gateway.
2

Check it with the CLI

# Show the gateway's identities: the type, and the system-assigned identity's principal and tenant IDs
echo "๐Ÿ‘ค Identities of $GATEWAY:"
az aigateway identity show -g "$RG" -n "$GATEWAY" -o json

The output shows type, which includes SystemAssigned, and the system-assigned principalId and tenantId. If the command reports that no identity is assigned to the gateway, enable the system-assigned identity and grant the new principal Foundry User on the Foundry account. identity assign returns when the gateway is ready again:

# Turn on the gateway's system-assigned identity, and wait until the gateway is ready again
az aigateway identity assign -g "$RG" -n "$GATEWAY" --system-assigned true -o none &&
  echo "๐Ÿ‘ค The system-assigned identity of $GATEWAY is on"

# Read the principal ID of the system-assigned identity
PRINCIPAL_ID="$(az aigateway show -g "$RG" -n "$GATEWAY" --query identity.principalId -o tsv)"
echo "๐Ÿ‘ค Gateway identity: $PRINCIPAL_ID"

# Give the identity the Foundry User role on the Foundry account, so that the gateway can call its models
az role assignment create --assignee-object-id "$PRINCIPAL_ID" --assignee-principal-type ServicePrincipal \
  --role "53ca6127-db72-4b80-b1b0-d745d6d5456d" --scope "$FOUNDRY_ID" -o none &&
  echo "๐Ÿ” The gateway's identity has the Foundry User role on the Foundry account"
Warning

Don't turn off the system-assigned identity while assets use it. Disabling it recycles the service principal: every backend configured with it fails until you enable it again, and a re-enabled identity gets a new principal ID without the old role assignments.

Create a user-assigned identity and give it access

Pick one track. Each track creates the identity, attaches it to the gateway, and grants it Foundry User on the Foundry account.

Choose your track
1

Create and attach the identity

  1. In Managed identitiesConfigure identities, under User-assigned identities, select Add identity.
  2. Select Create new and, in Name, replace the default, the gateway's name, with the name from $IDENTITY, for example id-aigateway-backends. The portal creates the identity in the gateway's subscription, resource group, and region, which it shows below the name.
  3. Select Add identity. The new identity appears in the table with its Principal and Client ID.

To reuse an identity that already exists, select Use existing instead, then pick its Subscription and User-assigned managed identity.

2

Read the identity's IDs

You need the principal ID to grant access and the client ID to point the provider at the identity.

# Read the identity's IDs: the resource ID to attach it, the principal ID to grant it access, and the client ID
# to point the provider at it
export IDENTITY_ID="$(az identity show -g "$RG" -n "$IDENTITY" --query id -o tsv)"
export IDENTITY_CLIENT_ID="$(az identity show -g "$RG" -n "$IDENTITY" --query clientId -o tsv)"
export IDENTITY_PRINCIPAL_ID="$(az identity show -g "$RG" -n "$IDENTITY" --query principalId -o tsv)"
echo "๐Ÿ†” Identity: $IDENTITY_ID"
echo "๐Ÿ‘ค Client ID: $IDENTITY_CLIENT_ID"
echo "๐Ÿ‘ค Principal ID: $IDENTITY_PRINCIPAL_ID"
3

Grant Foundry User on the Foundry account

Attaching an identity doesn't grant it any access. In the Azure portal, open the Foundry account and select Access control (IAM)AddAdd role assignment. Select Foundry User, assign access to Managed identity, select your identity as the member, then select Review + assign.

Or grant the role from the command line:

# Give your identity the Foundry User role on the Foundry account, so that it can call the account's models
az role assignment create --assignee-object-id "$IDENTITY_PRINCIPAL_ID" --assignee-principal-type ServicePrincipal \
  --role "53ca6127-db72-4b80-b1b0-d745d6d5456d" --scope "$FOUNDRY_ID" -o none &&
  echo "๐Ÿ” Identity $IDENTITY has the Foundry User role on the Foundry account"
Checkpoint

User-assigned identities lists your identity, and the Foundry account's Access control (IAM)Role assignments shows it with Foundry User.

Important

Role assignments can take several minutes to take effect. If you switch the provider before then, model calls fail with 403 from the backend until the assignment propagates.

Point the model provider at the identity

The provider still uses the system-assigned identity. Setting clientId in its managed identity settings switches it to your identity. Clients don't change: they keep calling the same alias with the same key or token.

Choose your track
1

Update the provider's authentication

The portal picks the identity only when you import a Foundry account: the Authentication method step offers Managed identity with a choice of the system-assigned identity or any attached user-assigned identity, and grants the selected identity Foundry User. To switch a provider that already exists, run this Azure CLI command, for example in Azure Cloud Shell. It sets clientId in the provider's managed identity settings and keeps the rest:

# Set clientId to your identity's client ID; the CLI merges it into the provider's other settings
echo "๐Ÿ”Œ Authentication of $PROVIDER:"
az aigateway model-provider update -g "$RG" --gateway-name "$GATEWAY" -n "$PROVIDER" \
  --foundry "{authentication:{managedIdentity:{clientId:$IDENTITY_CLIENT_ID}}}" \
  --query properties.authentication
Checkpoint

The CLI output shows a managedIdentity with clientId equal to $IDENTITY_CLIENT_ID and resource set to https://cognitiveservices.azure.com/: the provider's properties.authentication.managedIdentity.clientId.

Verify

1

Confirm which identity the provider uses

# Read the client ID that the provider signs in with, and compare it with your identity's client ID
PROVIDER_CLIENT_ID="$(az aigateway model-provider show -g "$RG" --gateway-name "$GATEWAY" -n "$PROVIDER" \
  --query properties.authentication.managedIdentity.clientId -o tsv)"
echo "๐Ÿ”Œ Provider's client ID: $PROVIDER_CLIENT_ID"
echo "๐Ÿ‘ค Identity's client ID: $IDENTITY_CLIENT_ID"

Both lines print the same client ID. In the portal, Managed identitiesView access now lists your provider under your identity's name instead of System-assigned. The list reflects the configured binding, not the live role assignment on the Foundry account, so a missing role still shows up only as a failed call.

2

Call the model through the gateway

# Call the alias through the gateway with your runtime key, and print only the HTTP status code
echo "๐Ÿค– HTTP status of a call to $AI_GATEWAY_MODEL through the gateway:"
curl -sS -o /dev/null -w "HTTP %{http_code}\n" "${AI_GATEWAY_URL%/}/default/models/openai/v1/chat/completions" \
  -H "Authorization: Bearer $AI_GATEWAY_API_KEY" -H "Content-Type: application/json" \
  -d "{\"model\":\"$AI_GATEWAY_MODEL\",\"messages\":[{\"role\":\"user\",\"content\":\"ping\"}]}"

Expect HTTP 200. A 403 that mentions permissions on the Foundry account means the role assignment hasn't propagated yet: wait a few minutes and retry.

You called the same alias with the same key as in lab 02. Only the gateway's outbound credential changed, and no backend key was stored or sent.

Checkpoint

The provider names your user-assigned identity, View access shows the new binding, and the model still answers with 200.

Go further

Use stored credentials when a backend can't take a managed identity

Managed identity needs a backend that accepts Microsoft Entra tokens. When it doesn't, store the backend's own credential in the gateway, never in the calling app.

BackendHow the gateway authenticates
Microsoft Foundry and Azure OpenAIManaged identity (recommended) or API key, chosen in the import's Authentication method step
OpenAI, Anthropic, AWS Bedrock, Google Vertex AI, and other custom modelsThe provider's API key, entered when you add the custom model (lab 03)
MCP server backendsNone, API key, OAuth 2.0, or Managed identity, set per backend (lab 04)
  • Prefer managed identity whenever the backend supports it.
  • The gateway stores backend keys and tokens with the asset or connection and never returns them to clients.
  • Rotate backend credentials on their own schedule. They're independent of the runtime keys callers use.

At API version 2026-11-01, a provider stores its key as one header in authentication.headers, in place of managedIdentity, for example "authentication": { "headers": [ { "name": "api-key", "value": "<foundry-key>" } ] } for Foundry. A read returns only the header's name, never its value. So a script that reads a provider and writes it back must send the key again, or the provider loses it.

Call MCP backends as the signed-in user

A managed identity calls a backend as the gateway, so every caller gets the gateway's access. Some MCP backends, such as an expenses API, must instead see who the user is and return only that user's data. For those, an MCP server backend can use OAuth 2.0 with the On behalf of profile: the client signs the user in with a Microsoft Entra token for the gateway, and the gateway exchanges it for a delegated token for the backend. A token exchange app registration holds the delegated permissions, and a user-assigned identity attached to the gateway, like the one in this lab, authenticates that app through a federated credential, so no client secret is stored. See Access MCP server backends with user credentials. 33 ยท Connect Microsoft 365 Copilot to MCP tools as the signed-in user builds the whole chain, from Copilot Cowork to an MCP server on Azure Container Apps.

Drop access the system identity no longer needs

After every asset that calls the Foundry account uses the user-assigned identity, the system-assigned identity's Foundry User assignment on that account is unused. Check View accessBackend access first: remove the assignment only if no other asset lists System-assigned for that account, because later labs in this series use it. A call that still returns 200 a few minutes after you remove it also proves that the provider uses the user-assigned identity.

Clean up

Pick the track you followed.

Choose your track

Keep the system-assigned identity and its Foundry User role. Later labs use them. Remove the user-assigned identity in this order, so the provider never points at an identity it can't use.

1

Switch the provider back to the system-assigned identity

Removing clientId selects the system-assigned identity again. A null value removes a setting:

# Remove clientId (null removes a setting), so the provider signs in with the system-assigned identity again
echo "๐Ÿ”Œ Authentication of $PROVIDER:"
az aigateway model-provider update -g "$RG" --gateway-name "$GATEWAY" -n "$PROVIDER" \
  --foundry "{authentication:{managedIdentity:{clientId:null}}}" \
  --query properties.authentication

The output no longer shows clientId.

2

Detach the identity from the gateway

Remove your identity from the gateway. identity remove returns when the gateway is ready again:

# Detach your identity from the gateway, and wait until the gateway is ready again
az aigateway identity remove -g "$RG" -n "$GATEWAY" --user-assigned "$IDENTITY_ID" -o none &&
  echo "๐Ÿšฎ Identity $IDENTITY is detached from $GATEWAY"

This detaches only your identity. The system-assigned identity and any other user-assigned identities stay attached.

In the portal, select the identity's checkbox in User-assigned identities, select the trash icon in the selection bar, then Detach. The confirmation says that detaching removes the identity from this gateway only: the identity and its role assignments stay.

3

Remove the role assignment and delete the identity

# Remove your identity's Foundry User role assignment from the Foundry account
az role assignment delete --assignee "$IDENTITY_PRINCIPAL_ID" --role "53ca6127-db72-4b80-b1b0-d745d6d5456d" --scope "$FOUNDRY_ID" &&
  echo "๐Ÿšฎ The Foundry User role assignment of $IDENTITY is deleted"

# Delete the identity, now that nothing uses it
az identity delete -g "$RG" -n "$IDENTITY" &&
  echo "๐Ÿšฎ Identity $IDENTITY is deleted"

Delete the role assignment first. Deleting the identity alone leaves an orphaned assignment on the account.

Checkpoint

The model call from Verify still returns 200, and View access shows the provider under System-assigned again.

azd down and terraform destroy don't touch your gateway from lab 02 or its system-assigned identity, because the Bicep + azd and Terraform tracks created their own gateway.

Next steps

Finished the lab?

Mark it complete to track your progress across paths. It's saved only in this browser.