Azure AI Gateway Labs
Documentation Open portal
Enterprise readiness · Lab 21

Deploy an enterprise reference architecture

Deploy a production-shaped Azure AI Gateway environment from modular Bicep or Terraform, with a keyless Foundry backend, models and MCP tools behind RBAC, a key per app, layered policies, and monitoring. Then promote it from dev to prod with a GitHub Actions workflow that signs in with OpenID Connect and smoke-tests every deployment.

Advanced 1 h BicepTerraform Updated
Source Edit

What you'll do

  • Explain the identity, policy, observability, and naming decisions behind a production AI Gateway deployment
  • Deploy a modular, subscription-scope Bicep template or Terraform configuration, with separate settings files for dev and prod
  • Layer gateway-wide baseline policies, applied by a merge-safe script, on top of model policies declared in code
  • Automate preview, deployment, baselines, and a smoke test in GitHub Actions with OpenID Connect
  • Verify the model and MCP tool paths end to end, and clean up every resource

You've built every piece by hand in earlier labs: a keyless Foundry backend, aliases, MCP servers, per-caller limits, a gateway-wide baseline, and monitoring. Now the platform team wants the whole setup as code. Dev and prod should differ only in their parameters, every change should go through a pull request, and a pipeline should deploy it without a stored secret. In this lab you deploy a modular reference architecture, apply the baseline, prove it works with a smoke test, and wire it all into GitHub Actions.

Before you begin

One-time setupThe Setup guide tasks that this lab uses. Mark each one done once, and the labs stop listing it.

  • Complete Set gateway-wide policy baselines, Monitor usage, cost, and traces, and Call backends with managed identities. This lab builds on what you learned there, not on the resources you created: it deploys a new, separate environment.
  • For the Terraform track: Terraform 1.9 or later, as in Deploy a lab with Terraform.
  • The Azure CLI installs Bicep the first time you deploy. The telemetry script and the smoke test read the gateway with the aigateway extension. Its az aigateway policy commands take each setting as a flag, not a policy array, so to apply the whole baseline from a file, the baseline script calls the management API with az rest.
  • Owner, or Contributor plus Role Based Access Control Administrator, on the subscription. The template creates two resource groups and several role assignments.
  • Quota for gpt-5-mini with the GlobalStandard SKU in the region you deploy to. The dev environment uses 50 capacity units (50,000 tokens per minute).
  • For Automate deployments with GitHub Actions: a GitHub repository where you can create environments and variables, the GitHub CLI signed in with gh auth login, and permission to create an app registration in Microsoft Entra ID.

Run every command from this lab's folder, ~/aigateway/labs/21-enterprise-reference, so the infra/ paths resolve. The Terraform track moves into terraform/ to deploy, then back. Set these variables once:

# Set the region and the deployment name that the template and the rest of the lab use
export LOCATION="eastus2"            # deployment metadata, the Foundry account, and the monitoring resources
export ENV_NAME="aigw-dev"           # deployment name, the same as environmentName in main.bicepparam
echo "📝 Deployment $ENV_NAME in $LOCATION"

# Read your own object ID: main.bicepparam, or TF_VAR_principal_id on the Terraform track, gives you data-plane
# access to the Foundry account, the alias, and the MCP server
export AZURE_PRINCIPAL_ID="$(az ad signed-in-user show --query id -o tsv)"
echo "👤 Your principal ID: $AZURE_PRINCIPAL_ID"

# Show the subscription that you deploy to, so that you can check it before you go on
echo "🆔 Subscription that you deploy to:"
az account show --query "{subscription:name, id:id}" -o table

How it works

One subscription-scope template, infra/main.bicep, creates two resource groups and composes small modules. Each module owns one concern, so the people who own that concern can review and change it on their own, and no module outputs a secret. Parameter files hold everything that differs between environments. The Terraform track has the same shape: a root module, terraform/main.tf, calls one module in terraform/modules/ per Bicep module, and dev.tfvars and prod.tfvars replace the parameter files.

Gateway-wide policies aren't in Bicep or Terraform. They live on the gateway's default workspace, and declaring that workspace in a template would replace it. Instead, a script merges them in after each deployment, as in Set gateway-wide policy baselines.

FileResource groupDeploys
modules/monitoring.bicep…-gatewayLog Analytics workspace and a workspace-based Application Insights component, <gateway>-insights, with OTLP ingestion on
modules/foundry.bicep…-foundrymodelsFoundry account with key authentication turned off, and the gpt-5-mini deployment
modules/gateway.bicep…-gatewayThe AI Gateway (Microsoft.ApiManagement/aigateways, system-assigned identity) and Foundry User for the identity on the Foundry account
modules/models.bicep…-gatewayA Foundry model provider that authenticates with the managed identity, the backend model with a token limit and content safety, the alias chat, and AI Gateway Models User on the alias
modules/tools.bicep…-gatewayThe Microsoft Learn MCP server learn, and AI Gateway Tools User on it
modules/keys.bicep…-gatewayOne runtime API key per app: web-chat and batch-jobs
modules/telemetry.bicep…-gatewayThe gateway's OpenTelemetry exporter to the component, with the recommended token claims. scripts/connect-telemetry.sh deploys it
scripts/connect-telemetry.shNot applicableWaits for the component's OTLP endpoints, grants the gateway's identity Monitoring Metrics Publisher on its Data Collection Rule, then deploys modules/telemetry.bicep
scripts/apply-baselines.shNot applicableGateway scope Request rate limit and Cost limit from baselines.json

On the Terraform track, each module is a folder with the same name, such as terraform/modules/gateway. The telemetry module also does what connect-telemetry.sh does, so terraform apply connects telemetry in the same run. Both tracks apply the baseline with scripts/apply-baselines.sh.

Make the architecture decisions

Each decision below is already built into the templates. In your own environment, change the parameters first, and change the shape only when your organization needs something different.

Identity: no secrets between services or in the pipeline

Each trust boundary uses its own mechanism, and none of them needs a stored secret:

BoundaryMechanismWhere
Gateway to FoundryThe gateway's system-assigned identity has Foundry User on the account. The account sets disableLocalAuth: true, so Foundry keys don't work at all.gateway.bicep, foundry.bicep
Apps to gatewayOne runtime API key per app, so every app gets its own policy counters, its own row in Monitoring, and its own rotation. Or keyless Microsoft Entra calls through AI Gateway Models User on the alias and AI Gateway Tools User on the MCP server, granted to groups.keys.bicep, models.bicep, tools.bicep
Pipeline to AzureAn OpenID Connect federated credential. GitHub exchanges a short-lived token for an Azure token, so there's no client secret to rotate or leak.deploy-workflow.yml

Roles are granted at the narrowest scope that works: the alias instead of the whole gateway, and one MCP server instead of every tool. Templates output key names only, never key values. The smoke test reads a key with az aigateway api-key list-secrets when it needs one.

Policies: a gateway floor plus model limits

LayerDeclared inPoliciesCounted per
Gatewaybaselines.json, applied by scripts/apply-baselines.shRequest rate limit: 120 calls per 60 seconds, on models and MCP servers. Cost limit: $20 per day, one budget across all models.Caller identity
Modelmodels.bicepToken rate limit: 20,000 tokens per minute in dev. Content safety: Balanced (Medium) for all four categories.Caller identity

The gateway evaluates every applicable policy before it forwards a request, and any one of them can block it. The platform team owns the gateway floor, and app teams tune the model layer through parameters. Blocked calls get 429 from a rate limit, 403 with LlmCostQuotaExceeded from the cost limit, and 400 from content safety.

The baseline script reads the workspace's policies array, updates policies of the same type in place, appends missing ones, and writes the array back with PATCH. Updating in place keeps a cost limit's id, its portal-only overrides, and the spend it has already counted.

Note

The portal writes "scope": "workspace" on Gateway scope rate and cost limits, so baselines.json does too. If the API rejects a field, create the policy once in the portal at Gateway scope, read the Gateway scope policies with az aigateway policy list -g "$RG" --gateway-name "$GATEWAY", and copy that shape into baselines.json.

Observability: the destination in Bicep, the exporter after it

monitoring.bicep creates the Log Analytics workspace and the Application Insights component, with retention set per environment, and turns on OTLP ingestion. A few minutes after the deployment, Azure creates the component's OTLP endpoints and a Data Collection Rule in a managed resource group. The gateway's OpenTelemetry exporter, in telemetry.bicep, needs those endpoints, and the gateway's managed identity needs Monitoring Metrics Publisher on the rule before the exporter is created. Bicep can't wait for the endpoints, and it can't assign the role, because the managed resource group has a deny assignment that blocks deployments. So scripts/connect-telemetry.sh runs after each deployment: it waits for the endpoints, assigns the role with the Azure CLI, then deploys telemetry.bicep. The result matches the portal's Configure telemetry wizard: metrics, logs, and traces, with the recommended token claims.

Terraform can wait, so on the Terraform track the telemetry module does the same work inside terraform apply. It runs the same wait with the Azure CLI, reads the endpoints once and keeps them in the state, assigns the role, then creates the exporter.

The exporter sends to the component's OTLP endpoints and doesn't link to the Application Insights resource, yet the Monitoring dashboard still reads its data. The other exporter type, ApplicationInsights, which the reference sample uses, is legacy: the portal flags it because it no longer activates the OTLP signals the dashboard needs. An exporter's type can't change, so the script replaces an ApplicationInsights exporter that an earlier deployment created.

Naming, tags, and resource groups

  • Deterministic names. A resource token, the first eight characters of uniqueString(subscription, environmentName, location), goes into every name. Deploying again updates the same resources instead of creating new ones.
  • Two resource groups. rg-<env>-<token>-gateway holds the gateway and its monitoring, and rg-<env>-<token>-foundrymodels holds model capacity. You can secure, scale, and delete them separately, and only the gateway's identity needs access to Foundry.
  • A unique gateway name. The gateway is aigw-<token>. Its runtime URL starts with the name's first 10 characters and a hash, like https://aigw-abcde-k3v7qz2m9xw4p.eastus2-01.ai.gateway.azure.com. Set aiGatewayName to choose your own: up to 24 letters, digits, and hyphens, starting with a letter.
  • Tags. Resource groups and top-level resources get environment and workload: ai-gateway, plus the tags from the parameter file, such as costCenter.
  • Regions. aiGatewayLocation defaults to eastus2. Set it to any region where AI Gateway is available, as in Find the available regions. The other resources follow the deployment's --location, so pick one where the model is available.
  • On the Terraform track, the token is six characters of a hash of the subscription, the environment name, and the location. The resource groups are rg-<env>-gateway and rg-<env>-foundrymodels, the environment tag is environment-name, and the gateway's region, ai_gateway_location, defaults to location, so one region drives every resource.

Environments: one template, two parameter files

Settingmain.bicepparam (dev)prod.bicepparam (prod)
environmentNameaigw-devaigw-prod
Developer access (principalId)You, from AZURE_PRINCIPAL_IDNone
Model capacity (modelCapacity)50200
Tokens per minute per caller (tokensPerMinutePerCaller)20,00050,000
Microsoft Entra access (modelUsers, toolUsers)Your user onlyGroups you add
Log retention (logRetentionInDays)30 days90 days
costCenter tagdevplatform

The Terraform track's dev.tfvars and prod.tfvars set the same values, with the variable names in snake case, such as model_capacity and tokens_per_minute_per_caller. Their environment names are aigw-dev-tf and aigw-prod-tf, so a Terraform deployment never touches a Bicep one.

Tip

Put prod in its own subscription so quotas, role assignments, and Azure Policy stay isolated from dev. The template is subscription-scoped, so it works unchanged. Use a separate deployment identity for each subscription.

Delivery: GitHub Actions with OpenID Connect

The workflow installs the aigateway extension, previews every change with what-if, deploys, connects telemetry, applies the baseline, and runs the smoke test. A prod GitHub environment with required reviewers gates production, and a concurrency group per environment stops two runs from deploying to the same environment at once. The deployment identity needs Contributor and Role Based Access Control Administrator on the subscription, because the template creates role assignments. To narrow that, you can add a condition that only lets it assign the roles the template needs, as described in Delegate Azure access management to others. With Terraform, the same workflow runs terraform plan and terraform apply instead of what-if and the deployment, and keeps the Terraform state in an Azure Storage container that every run shares.

Review the templates

Choose your track

main.bicep wires the modules together and passes outputs from one to the next, so Bicep deploys them in the right order: monitoring and Foundry first, then the gateway, then models, tools, and keys.

infra/main.bicepGitHub
// Enterprise reference deployment for Azure AI Gateway. One subscription-scope template creates two resource groups
// and composes small modules:
//
//   rg-<env>-<token>-foundrymodels   foundry.bicep         Foundry account (keyless) + model deployment
//   rg-<env>-<token>-gateway         monitoring.bicep      Log Analytics + Application Insights with OTLP ingestion
//                                    gateway.bicep         aigateways resource + Foundry User role for the gateway identity
//                                    models.bicep          Foundry provider + backend model (tokenLimit, contentSafety) + alias
//                                    tools.bicep           Microsoft Learn MCP server
//                                    keys.bicep            one runtime API key per app
//
// After deployment, scripts/connect-telemetry.sh (or .ps1) connects the gateway to Application Insights with
// telemetry.bicep, an OpenTelemetry exporter, once Azure has created the OTLP endpoints. Then
// scripts/apply-baselines.sh (or .ps1) applies the gateway-wide baseline policies.
targetScope = 'subscription'

import { principal } from './modules/types.bicep'

@minLength(1)
@maxLength(64)
@description('Environment name, for example aigw-dev or aigw-prod. Used in resource group names, tags, and the deterministic resource token.')
param environmentName string

@description('Region for the resource groups, the Foundry account, and the monitoring resources. The model and SKU must be available here.')
param location string = deployment().location

@description('Region for the gateway. Pick one where AI Gateway is available. List them with az provider show --namespace Microsoft.ApiManagement (Setup guide, Find the available regions).')
param aiGatewayLocation string = 'eastus2'

@description('Optional Microsoft Entra object ID of a developer. When set, the developer gets data-plane roles on the Foundry account, the alias, and the MCP server. Leave empty in production.')
param principalId string = ''

@description('Extra tags merged into the default tags on every resource group and resource.')
param tags object = {}

@description('Gateway name, max 24 characters. Defaults to aigw-<token>. Its first 10 characters start the runtime host name.')
param aiGatewayName string = ''

@description('Foundry model name to deploy.')
param modelName string = 'gpt-5-mini'

@description('Foundry model version to deploy.')
param modelVersion string = '2025-08-07'

@description('ARM-safe name of the Foundry deployment. It is also the backend model name in the gateway.')
param modelDeploymentName string = 'gpt-mini-latest'

@description('Model format for the Foundry deployment.')
param modelFormat string = 'OpenAI'

@description('Model deployment SKU name.')
param modelSkuName string = 'GlobalStandard'

@description('Deployment capacity, in thousands of tokens per minute for Global Standard.')
@minValue(1)
param modelCapacity int = 50

@description('Tokens per minute that each caller identity (API key or Microsoft Entra principal) can use on the backend model.')
@minValue(1)
param tokensPerMinutePerCaller int = 20000

@allowed([
  'None'
  'Low'
  'Medium'
  'High'
])
@description('Content safety threshold for every category on the backend model. The threshold is the lowest severity that is blocked: Low is strictest, None turns the category off.')
param contentSafetySeverity string = 'Medium'

@description('Client-facing alias that apps send in the request "model" field.')
param aliasName string = 'chat'

@description('Apps that get their own runtime API key. Each key is named after the app.')
param appNames string[] = [
  'web-chat'
  'batch-jobs'
]

@description('Microsoft Entra principals (usually groups) that get the AI Gateway Models User role on the alias.')
param modelUsers principal[] = []

@description('Microsoft Entra principals (usually groups) that get the AI Gateway Tools User role on the Microsoft Learn MCP server.')
param toolUsers principal[] = []

@description('Days to keep gateway telemetry in Log Analytics.')
@minValue(30)
@maxValue(730)
param logRetentionInDays int = 30

var resourceToken = take(toLower(uniqueString(subscription().id, environmentName, location)), 8)
var environmentLabel = toLower(environmentName)
var allTags = union({
  environment: environmentName
  workload: 'ai-gateway'
}, tags)
var gatewayResourceGroupName = 'rg-${environmentLabel}-${resourceToken}-gateway'
var foundryModelsResourceGroupName = 'rg-${environmentLabel}-${resourceToken}-foundrymodels'
var effectiveAiGatewayName = !empty(aiGatewayName) ? aiGatewayName : 'aigw-${resourceToken}'
var modelServicesName = 'aimodels-${resourceToken}'
var appInsightsName = take('${effectiveAiGatewayName}-insights', 64)
var logAnalyticsWorkspaceName = take('${effectiveAiGatewayName}-logs', 63)
var developer = empty(principalId) ? [] : [
  {
    principalId: principalId
    principalType: 'User'
  }
]
var gatewayModelDeployments = [
  {
    deploymentName: modelDeploymentName
    modelName: modelName
    modelFormat: modelFormat
    modelVersion: modelVersion
    skuName: modelSkuName
    capacity: modelCapacity
    description: 'Default chat model for every app behind the gateway.'
  }
]

resource gatewayRg 'Microsoft.Resources/resourceGroups@2021-04-01' = {
  name: gatewayResourceGroupName
  location: location
  tags: allTags
}

resource foundryModelsRg 'Microsoft.Resources/resourceGroups@2021-04-01' = {
  name: foundryModelsResourceGroupName
  location: location
  tags: allTags
}

module monitoring './modules/monitoring.bicep' = {
  name: 'monitoring-${resourceToken}'
  scope: gatewayRg
  params: {
    location: location
    tags: allTags
    logAnalyticsWorkspaceName: logAnalyticsWorkspaceName
    appInsightsName: appInsightsName
    retentionInDays: logRetentionInDays
  }
}

module foundry './modules/foundry.bicep' = {
  name: 'foundry-models-${resourceToken}'
  scope: foundryModelsRg
  params: {
    location: location
    tags: allTags
    aiServicesName: modelServicesName
    principalId: principalId
    modelDeployments: gatewayModelDeployments
  }
}

module gateway './modules/gateway.bicep' = {
  name: 'ai-gateway-${resourceToken}'
  scope: gatewayRg
  params: {
    location: aiGatewayLocation
    gatewayName: effectiveAiGatewayName
    tags: allTags
    foundryResourceIds: [
      foundry.outputs.aiServicesId
    ]
  }
}

// Every module below takes gateway.outputs.gatewayName, so it starts only after the gateway and the Foundry User
// role assignment exist.
module models './modules/models.bicep' = {
  name: 'ai-gateway-models-${resourceToken}'
  scope: gatewayRg
  params: {
    gatewayName: gateway.outputs.gatewayName
    providerName: 'foundry'
    providerDisplayName: 'Foundry'
    providerDescription: 'Managed-identity Foundry provider for the ${environmentName} environment.'
    foundryEndpoint: foundry.outputs.aiServicesEndpoint
    foundryResourceIds: [
      foundry.outputs.aiServicesId
    ]
    foundryModels: [for (deployment, i) in gatewayModelDeployments: {
      armName: deployment.deploymentName
      description: deployment.description
      resourceId: foundry.outputs.modelDeploymentIds[i]
      modelName: deployment.modelName
      modelVersion: deployment.modelVersion
      tokenLimit: tokensPerMinutePerCaller
      supportedEndpoints: [
        '/openai/v1/chat/completions'
        '/openai/v1/responses'
      ]
    }]
    contentSafetySeverity: contentSafetySeverity
    aliasName: aliasName
    aliasUsers: concat(developer, modelUsers)
  }
}

module tools './modules/tools.bicep' = {
  name: 'ai-gateway-tools-${resourceToken}'
  scope: gatewayRg
  params: {
    gatewayName: gateway.outputs.gatewayName
    toolUsers: concat(developer, toolUsers)
  }
}

module keys './modules/keys.bicep' = {
  name: 'ai-gateway-keys-${resourceToken}'
  scope: gatewayRg
  params: {
    gatewayName: gateway.outputs.gatewayName
    appNames: appNames
  }
}

output AZURE_LOCATION string = location
output AZURE_TENANT_ID string = tenant().tenantId
output AI_GATEWAY_NAME string = gateway.outputs.gatewayName
output AI_GATEWAY_LOCATION string = aiGatewayLocation
output AI_GATEWAY_RESOURCE_GROUP string = gatewayRg.name
output AI_GATEWAY_RESOURCE_ID string = gateway.outputs.gatewayId
output AI_GATEWAY_URL string = gateway.outputs.gatewayUrl
output AI_GATEWAY_PRINCIPAL_ID string = gateway.outputs.gatewayPrincipalId
output AI_GATEWAY_MODEL string = models.outputs.aliasName
output MCP_ENDPOINT string = tools.outputs.mcpEndpoint
output API_KEY_NAMES array = keys.outputs.apiKeyNames
output FOUNDRY_MODELS_RESOURCE_GROUP string = foundryModelsRg.name
output FOUNDRY_MODELS_NAME string = foundry.outputs.aiServicesName
output FOUNDRY_MODELS_RESOURCE_ID string = foundry.outputs.aiServicesId
output APPLICATION_INSIGHTS_NAME string = monitoring.outputs.appInsightsName
output LOG_ANALYTICS_WORKSPACE_ID string = monitoring.outputs.logAnalyticsWorkspaceId

On GitHub, open infra/main.bicep.

gateway.bicep is the only module that declares the gateway itself. The others reference it as existing and only add child resources to it.

infra/modules/gateway.bicepGitHub
// The AI Gateway itself: a Microsoft.ApiManagement/aigateways resource with a system-assigned managed identity, in
// the default SKU (sku is omitted, as in the reference sample), and the Foundry User role for the gateway identity
// on every Foundry account it calls.
targetScope = 'resourceGroup'

@description('Location of the gateway. Pick one where AI Gateway is available. List them with az provider show --namespace Microsoft.ApiManagement (Setup guide, Find the available regions).')
param location string

@description('Gateway name, max 24 characters. Its first 10 characters start the runtime host name.')
@maxLength(24)
param gatewayName string

@description('Tags applied to the gateway.')
param tags object = {}

@description('Optional subnet resource ID for outbound virtual network integration. The subnet must be delegated to Microsoft.App/environments. You can add, change, or remove it after the gateway is created.')
param backendSubnetId string = ''

@description('Resource IDs of the Foundry accounts the gateway calls with its managed identity.')
param foundryResourceIds array = []

#disable-next-line BCP081
resource aiGateway 'Microsoft.ApiManagement/aigateways@2026-11-01' = {
  name: gatewayName
  location: location
  tags: tags
  identity: {
    type: 'SystemAssigned'
  }
  properties: empty(backendSubnetId) ? {} : { backend: { subnet: { id: backendSubnetId } } }
}

module foundryUserRoleAssignments './foundry-user-role.bicep' = [for (foundryResourceId, i) in foundryResourceIds: {
  name: 'foundry-rbac-${i}-${uniqueString(foundryResourceId, gatewayName)}'
  scope: resourceGroup(split(foundryResourceId, '/')[2], split(foundryResourceId, '/')[4])
  params: {
    foundryAccountName: last(split(foundryResourceId, '/'))
    gatewayPrincipalId: aiGateway.identity.principalId
    gatewayResourceId: aiGateway.id
  }
}]

output gatewayName string = aiGateway.name
output gatewayId string = aiGateway.id
output gatewayUrl string = 'https://${aiGateway.properties.frontend.defaultHostname}'
output runtimeHostname string = aiGateway.properties.frontend.defaultHostname
output gatewayPrincipalId string = aiGateway.identity.principalId
output foundryUserRoleAssignmentNames array = [for (foundryResourceId, i) in foundryResourceIds: foundryUserRoleAssignments[i].outputs.roleAssignmentName]

On GitHub, open infra/modules/gateway.bicep.

Warning

Each deployment sends a PUT for the gateway with exactly the properties in gateway.bicep. If you change gateway settings outside Bicep, add them to this module. Otherwise, the next deployment can reset them. Outbound virtual network integration and Public network access are the exceptions: a deployment that doesn't set them keeps the gateway's current values. gateway.bicep accepts the integration subnet as backendSubnetId, but main.bicep doesn't pass it. To use it, add a parameter to main.bicep, or set it in the portal under Networking or with az aigateway network update --subnet. See Lock down backend traffic with private networking.

models.bicep declares the model-level policies right on the backend model, so they're versioned with it:

infra/modules/models.bicepGitHub
// Publishes Foundry model deployments through the gateway: one Foundry model provider that authenticates with the
// gateway's managed identity, one backend model per deployment with model-level policies (a per-caller token limit
// and content safety), a stable client-facing alias, and optional AI Gateway Models User role assignments on that
// alias for Microsoft Entra callers.
targetScope = 'resourceGroup'

import { principal } from './types.bicep'

@description('Name of the existing gateway in this resource group.')
param gatewayName string

@description('Name of the model provider to create.')
param providerName string = 'foundry'

@description('Display name of the model provider.')
param providerDisplayName string = 'Foundry'

@description('Description of the model provider.')
param providerDescription string = ''

@description('Endpoint of the Foundry account, for example https://<foundry-account-name>.cognitiveservices.azure.com/.')
param foundryEndpoint string

@description('Resource IDs of the Foundry accounts behind the provider.')
param foundryResourceIds array

@description('Audience the gateway requests a managed identity token for when it calls Foundry.')
param foundryManagedIdentityResource string = 'https://cognitiveservices.azure.com/'

@description('Backend models: objects with armName, description, resourceId, modelName, modelVersion, tokenLimit, and supportedEndpoints.')
param foundryModels array

@description('Content safety threshold for every category on every backend model. The threshold is the lowest severity that is blocked.')
@allowed([
  'None'
  'Low'
  'Medium'
  'High'
])
param contentSafetySeverity string = 'Medium'

@description('Client-facing alias that apps send in the request "model" field.')
param aliasName string = 'chat'

@description('Index in foundryModels of the backend model the alias points to.')
@minValue(0)
param aliasModelIndex int = 0

@description('Microsoft Entra principals that get the AI Gateway Models User role on the alias.')
param aliasUsers principal[] = []

var normalizedFoundryEndpoint = endsWith(foundryEndpoint, '/') ? foundryEndpoint : '${foundryEndpoint}/'
var aiGatewayModelsUserRoleDefinitionId = 'a6b68e3d-a1f0-4c60-8678-237c050cf4ea'

#disable-next-line BCP081
resource aiGateway 'Microsoft.ApiManagement/aigateways@2026-11-01' existing = {
  name: gatewayName
}

#disable-next-line BCP081
resource defaultWorkspace 'Microsoft.ApiManagement/aigateways/workspaces@2026-11-01' existing = {
  parent: aiGateway
  name: 'default'
}

#disable-next-line BCP081
resource foundryProvider 'Microsoft.ApiManagement/aigateways/workspaces/modelProviders@2026-11-01' = {
  parent: defaultWorkspace
  name: providerName
  properties: {
    modelProviderType: 'Foundry'
    displayName: providerDisplayName
    description: empty(providerDescription) ? null : providerDescription
    foundry: {
      endpoint: {
        url: normalizedFoundryEndpoint
      }
      resourceIds: foundryResourceIds
    }
    authentication: {
      managedIdentity: {
        resource: foundryManagedIdentityResource
      }
    }
  }
}

#disable-next-line BCP081
resource foundryCatalogModels 'Microsoft.ApiManagement/aigateways/workspaces/modelProviders/models@2026-11-01' = [for model in foundryModels: {
  parent: foundryProvider
  name: model.armName
  properties: {
    description: model.?description ?? null
    displayName: model.modelName
    supportedEndpoints: model.supportedEndpoints
    deployment: {
      resourceId: model.resourceId
      modelName: last(split(model.resourceId, '/'))
      modelVersion: model.?modelVersion ?? null
    }
    policies: [
      {
        type: 'tokenLimit'
        period: 'minute'
        count: model.?tokenLimit ?? 10000
        counterKey: [ 'identity' ]
      }
      {
        type: 'contentSafety'
        hateSeverity: contentSafetySeverity
        violenceSeverity: contentSafetySeverity
        sexualSeverity: contentSafetySeverity
        selfHarmSeverity: contentSafetySeverity
      }
    ]
  }
}]

#disable-next-line BCP081
resource alias 'Microsoft.ApiManagement/aigateways/workspaces/aliases@2026-11-01' = {
  parent: defaultWorkspace
  name: aliasName
  properties: {
    backendModels: [
      { resourceId: foundryCatalogModels[aliasModelIndex].id }
    ]
  }
}

resource aliasModelsUsers 'Microsoft.Authorization/roleAssignments@2022-04-01' = [for user in aliasUsers: {
  scope: alias
  name: guid(alias.id, user.principalId, aiGatewayModelsUserRoleDefinitionId)
  properties: {
    principalId: user.principalId
    principalType: user.principalType
    roleDefinitionId: subscriptionResourceId('Microsoft.Authorization/roleDefinitions', aiGatewayModelsUserRoleDefinitionId)
  }
}]

output providerName string = foundryProvider.name
output modelNames array = [for (model, i) in foundryModels: foundryCatalogModels[i].name]
output aliasName string = alias.name
output aliasId string = alias.id

On GitHub, open infra/modules/models.bicep.

The gateway floor is a plain policy file that goes through code review like any template:

infra/baselines.jsonGitHub
[
  {
    "type": "requestRateLimit",
    "count": 120,
    "period": "minute",
    "counterKey": ["identity"],
    "scope": "workspace"
  },
  {
    "type": "costLimit",
    "amount": 20,
    "period": "day",
    "counterKey": ["identity"],
    "scope": "workspace"
  }
]

On GitHub, open infra/baselines.json.

The other files are short. Open them on GitHub: monitoring.bicep, foundry.bicep, foundry-user-role.bicep, tools.bicep, keys.bicep, telemetry.bicep, types.bicep, main.bicepparam, and prod.bicepparam.

Deploy the dev environment

Deploy from your machine first, so you see each stage before you automate it.

Choose your track
1

Preview the changes

# Preview what the deployment would create or change, with the dev parameters, without changing anything
echo "📊 What deployment $ENV_NAME would change:"
az deployment sub what-if -n "$ENV_NAME" -l "$LOCATION" \
  -f infra/main.bicep -p infra/main.bicepparam

On a first run, the preview lists the two resource groups and the resources in them as new. On later runs, it shows only what your change would modify. Paste it into the pull request so reviewers see the effect, not just the diff.

2

Deploy

# Deploy the subscription-scope template with the dev parameters, and keep only its provisioning state
STATE="$(az deployment sub create -n "$ENV_NAME" -l "$LOCATION" \
  -f infra/main.bicep -p infra/main.bicepparam \
  --query properties.provisioningState -o tsv)" &&
  echo "🚀 Deployment $ENV_NAME finished: $STATE"

Provisioning the gateway and the model deployment takes about three minutes. The command prints 🚀 Deployment aigw-dev finished: Succeeded when everything is in place.

3

Load the outputs

Read the deployment outputs into the variables the scripts and the rest of the lab use:

# Read the deployment's outputs as JSON
OUTPUTS="$(az deployment sub show -n "$ENV_NAME" --query properties.outputs -o json)"

# Define a helper that reads one output: output names can come back with a different first-letter casing,
# so it matches them case-insensitively
out() { jq -r --arg k "$1" 'to_entries[] | select((.key | ascii_upcase) == $k) | .value.value' <<<"$OUTPUTS"; }

# Read the gateway, the alias, and the Foundry account into the variables that the scripts and later steps use
export RG="$(out AI_GATEWAY_RESOURCE_GROUP)"
export GATEWAY="$(out AI_GATEWAY_NAME)"
export AI_GATEWAY_URL="$(out AI_GATEWAY_URL)"
export AI_GATEWAY_MODEL="$(out AI_GATEWAY_MODEL)"
export AI_GATEWAY_LOCATION="$(out AI_GATEWAY_LOCATION)"
export AI_GATEWAY_PRINCIPAL_ID="$(out AI_GATEWAY_PRINCIPAL_ID)"
export FOUNDRY_RG="$(out FOUNDRY_MODELS_RESOURCE_GROUP)"
export FOUNDRY_NAME="$(out FOUNDRY_MODELS_NAME)"
export FOUNDRY_ID="$(out FOUNDRY_MODELS_RESOURCE_ID)"
echo "📝 Gateway $GATEWAY in $RG"
echo "🌐 Runtime URL: $AI_GATEWAY_URL"
echo "🔖 Model alias: $AI_GATEWAY_MODEL"
echo "🏭 Foundry account $FOUNDRY_NAME in $FOUNDRY_RG"

Expected output:

Text
📝 Gateway aigw-<token> in rg-aigw-dev-<token>-gateway
🌐 Runtime URL: https://aigw-abcde-k3v7qz2m9xw4p.eastus2-01.ai.gateway.azure.com
🔖 Model alias: chat
🏭 Foundry account aimodels-<token> in rg-aigw-dev-<token>-foundrymodels
4

Connect telemetry

Run the telemetry script. It waits until Azure has created the component's OTLP endpoints, grants the gateway's identity Monitoring Metrics Publisher on the Data Collection Rule, then deploys modules/telemetry.bicep. Right after the first deployment, the wait can take a few minutes.

# Wait for the OTLP endpoints, let the gateway's identity publish to them, then deploy the OpenTelemetry exporter
./infra/scripts/connect-telemetry.sh &&
  echo "📊 Telemetry is connected for $GATEWAY"

On GitHub, open infra/scripts/connect-telemetry.sh or infra/scripts/connect-telemetry.ps1.

Expected output:

Text
⏳ Waiting for the OTLP endpoints of aigw-<token>-insights
🆔 Data Collection Rule: managed-aigw-<token>-insights-dcr
🔐 The gateway can publish to the Data Collection Rule
📊 The gateway sends metrics, logs, and traces to aigw-<token>-insights
📊 Telemetry is connected for aigw-<token>

The role assignment needs the same rights as the deployment. Run the script after each deployment: running it again gives the same result.

On both tracks, apply the gateway baseline from the lab folder with the same script.

1

Preview the gateway baseline

Print the merged policy array without changing anything:

# Print the merged policy array without changing anything: with DRY_RUN=1, the script stops before it writes
DRY_RUN=1 ./infra/scripts/apply-baselines.sh &&
  echo "🔐 Baseline preview finished: the policies on $GATEWAY are unchanged"

The output shows a requestRateLimit entry and a costLimit entry with a newly generated id, plus any Gateway scope policies the gateway already has.

2

Apply the gateway baseline

# Merge baselines.json into the gateway's Gateway scope policies, write them back, and list the policy types
./infra/scripts/apply-baselines.sh &&
  echo "🔐 Gateway baseline is applied to $GATEWAY"
infra/scripts/apply-baselines.shGitHub
#!/usr/bin/env bash
# Applies the gateway-wide baseline policies in infra/baselines.json to an AI Gateway.
#
# Gateway-scope policies live in the policies array of the gateway's default workspace. A PATCH replaces the whole
# array, so the script reads the current array, merges the baselines into it, and writes it back:
#   - A baseline whose type already exists updates that policy in place and keeps its id, overrides, and any other
#     fields you set in the portal. Updating a costLimit this way keeps its accumulated spend.
#   - A baseline whose type doesn't exist is appended. A new costLimit gets a fresh, stable id.
#   - Policies of other types (for example ones you added in the portal) are left untouched.
# Running it twice gives the same result. The az aigateway policy commands take each setting as a flag, not a
# policy array, so to write the array from a file, the script calls the management API with az rest.
#
# Usage:
#   export RG="<gateway-resource-group>" GATEWAY="<gateway-name>"
#   ./apply-baselines.sh [baselines.json]
#   DRY_RUN=1 ./apply-baselines.sh          # print the merged array without changing anything
#
# Requires Azure CLI and jq.
set -euo pipefail

SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
BASELINES="${1:-$SCRIPT_DIR/../baselines.json}"
API_VERSION="2026-11-01"
: "${RG:?Set RG to the gateway resource group}"
: "${GATEWAY:?Set GATEWAY to the gateway name}"
command -v jq >/dev/null || { echo "This script needs jq." >&2; exit 1; }
jq -e 'type == "array" and all(.[]; has("type"))' "$BASELINES" >/dev/null \
  || { echo "$BASELINES must be a JSON array of policy objects." >&2; exit 1; }

SUB="$(az account show --query id -o tsv)"
URL="https://management.azure.com/subscriptions/$SUB/resourceGroups/$RG/providers/Microsoft.ApiManagement/aigateways/$GATEWAY/workspaces/default?api-version=$API_VERSION"
NEW_ID="$( (uuidgen 2>/dev/null || cat /proc/sys/kernel/random/uuid) | tr 'A-Z' 'a-z')"

CURRENT="$(az rest --method get --url "$URL")"
POLICIES="$(printf '%s' "$CURRENT" | jq -c --slurpfile base "$BASELINES" --arg id "$NEW_ID" '
  reduce $base[0][] as $b (.properties.policies // [];
    if any(.[]; .type == $b.type)
    then map(if .type == $b.type then . + $b else . end)
    else . + [if $b.type == "costLimit" and ($b.id // "") == "" then $b + {id: $id} else $b end]
    end)')"

echo "Gateway-scope policies after merge:"
printf '%s' "$POLICIES" | jq .

if [ "${DRY_RUN:-0}" = "1" ]; then
  echo "DRY_RUN=1: nothing was changed."
  exit 0
fi

az rest --method patch --url "$URL" --body "{\"properties\":{\"policies\":$POLICIES}}" --output none
echo "Applied. Current gateway-scope policy types:"
az rest --method get --url "$URL" --query "properties.policies[].type" -o tsv

On GitHub, open infra/scripts/apply-baselines.sh or infra/scripts/apply-baselines.ps1.

Expected output (end):

Text
Applied. Current gateway-scope policy types:
requestRateLimit
costLimit
🔐 Gateway baseline is applied to aigw-<token>

Run it again whenever baselines.json changes. It's idempotent: a second run gives the same array.

Checkpoint

On the Bicep track, the deployment printed 🚀 Deployment aigw-dev finished: Succeeded and the telemetry script printed 📊 Telemetry is connected. On the Terraform track, the apply printed Apply complete! Resources: 23 added, 0 changed, 0 destroyed. On both, RG and GATEWAY point at the new gateway, Monitoring shows the Overview, Models, Tools, Policies, and Traces tabs, and the baseline script lists requestRateLimit and costLimit.

Automate deployments with GitHub Actions

Copy this lab's infra/ folder to the root of your own repository, so the workflow finds infra/main.bicep. On the Terraform track, also copy the terraform/ folder. The workflow still runs the scripts and reads baselines.json from infra/. Then create an identity that GitHub Actions can sign in as, trust your repository's environments, and add the workflow. Learn more in Authenticate to Azure from GitHub Actions by OpenID Connect.

Set the repository once:

# Set the GitHub repository that holds the workflow, in the form org/repo
export GITHUB_REPO="<github-org>/<github-repo>"
echo "📝 GitHub repository: $GITHUB_REPO"
1

Create the deployment identity

Create an app registration and its service principal:

# Create the app registration that GitHub Actions signs in as, and keep its application (client) ID
export APP_ID="$(az ad app create --display-name "github-aigw-deploy" --query appId -o tsv)"
echo "👤 App registration github-aigw-deploy: $APP_ID"

# Create its service principal, the identity that gets the role assignments on your subscription
export SP_ID="$(az ad sp create --id "$APP_ID" --query id -o tsv)"
echo "👤 Service principal: $SP_ID"
2

Trust your GitHub environments

Add one federated credential per GitHub environment. The workflow's job declares environment:, so GitHub's token subject is the repository's subject prefix followed by :environment:<name>, and it must match exactly. The prefix is repo:<github-org>/<github-repo> for older repositories. Repositories created after July 15, 2026 use immutable subject claims, which add the owner and repository IDs, as in repo:<github-org>@<org-id>/<github-repo>@<repo-id>. GitHub returns the prefix that your repository uses, so read it instead of typing it.

# Read the subject prefix of your repository's OIDC tokens, with the immutable IDs if the repository uses them
export SUBJECT_PREFIX="$(gh api "repos/$GITHUB_REPO/actions/oidc/customization/sub" --jq .sub_claim_prefix)"
echo "🔖 Token subject prefix: $SUBJECT_PREFIX"

# For each GitHub environment, add a federated credential that trusts GitHub's tokens for that environment only
for ENV in dev prod; do
  # Write the credential to a file: its subject must match the workflow's environment exactly
  cat > "credential-$ENV.json" <<EOF
{
  "name": "github-$ENV",
  "issuer": "https://token.actions.githubusercontent.com",
  "subject": "$SUBJECT_PREFIX:environment:$ENV",
  "description": "GitHub Actions deployments to the $ENV environment",
  "audiences": [ "api://AzureADTokenExchange" ]
}
EOF
  # Add the credential to the app registration, then delete the file
  az ad app federated-credential create --id "$APP_ID" --parameters "credential-$ENV.json" -o none &&
    echo "👤 Federated credential github-$ENV trusts the $ENV environment of $GITHUB_REPO"
  rm "credential-$ENV.json"
done
3

Grant access to the subscription

# Read your subscription ID, the scope of the role assignments
export SUBSCRIPTION_ID="$(az account show --query id -o tsv)"
echo "🆔 Subscription: $SUBSCRIPTION_ID"

# Give the service principal Contributor, to deploy the resources, and Role Based Access Control Administrator,
# because the template creates role assignments
for ROLE in "Contributor" "Role Based Access Control Administrator"; do
  az role assignment create --assignee-object-id "$SP_ID" --assignee-principal-type ServicePrincipal \
    --role "$ROLE" --scope "/subscriptions/$SUBSCRIPTION_ID" -o none &&
    echo "🔐 The deployment identity has the $ROLE role on the subscription"
done
4

Create the GitHub environments

In your repository on GitHub, select SettingsEnvironmentsNew environment, and create dev and prod. On prod, turn on Required reviewers and add the people who approve production deployments. On the GitHub Free, Pro, and Team plans, required reviewers are only available for public repositories.

Then store the sign-in details as environment variables. They identify the identity but can't authenticate anyone on their own, so they're variables, not secrets.

# Read your tenant ID, the Microsoft Entra tenant that the workflow signs in to
TENANT_ID="$(az account show --query tenantId -o tsv)"
echo "🆔 Tenant: $TENANT_ID"

# Store the app's client ID, the tenant ID, and the subscription ID as variables of each GitHub environment
for ENV in dev prod; do
  gh variable set AZURE_CLIENT_ID --env "$ENV" --body "$APP_ID" -R "$GITHUB_REPO"
  gh variable set AZURE_TENANT_ID --env "$ENV" --body "$TENANT_ID" -R "$GITHUB_REPO"
  gh variable set AZURE_SUBSCRIPTION_ID --env "$ENV" --body "$SUBSCRIPTION_ID" -R "$GITHUB_REPO" &&
    echo "📝 The $ENV environment has AZURE_CLIENT_ID, AZURE_TENANT_ID, and AZURE_SUBSCRIPTION_ID"
done
Choose your track
1

Add the workflow

In your repository, copy the sample workflow into .github/workflows/, then commit and push both folders:

Bash
# Create the workflows folder, and copy the sample workflow into it
mkdir -p .github/workflows
cp infra/deploy-workflow.yml .github/workflows/deploy-ai-gateway.yml &&
  echo "📄 Workflow file: .github/workflows/deploy-ai-gateway.yml"
infra/deploy-workflow.ymlGitHub
# Sample GitHub Actions workflow for the enterprise reference deployment.
#
# Copy this file to .github/workflows/deploy-ai-gateway.yml in your own repository, with the lab's infra folder at the
# repository root (infra/main.bicep, infra/main.bicepparam, infra/prod.bicepparam, infra/scripts/...).
#
# It signs in to Azure with OpenID Connect (no client secret is stored in GitHub), installs the aigateway CLI extension
# that the scripts use, previews the change with what-if, deploys the subscription-scope template, connects the
# gateway's telemetry to Application Insights, applies the gateway-wide baseline policies, and runs the smoke test.
#
# One-time setup (see the lab for the commands):
#   - A Microsoft Entra app registration (or user-assigned managed identity) with a federated credential for each
#     GitHub environment: issuer https://token.actions.githubusercontent.com, audience api://AzureADTokenExchange,
#     subject <prefix>:environment:<dev|prod>. Read the prefix with
#     gh api repos/<org>/<repo>/actions/oidc/customization/sub --jq .sub_claim_prefix: repositories created after
#     July 15, 2026 add immutable IDs, as in repo:<org>@<org-id>/<repo>@<repo-id>.
#   - That identity needs Contributor and Role Based Access Control Administrator on the target subscription,
#     because the template creates resource groups and role assignments.
#   - GitHub environments named dev and prod (add required reviewers to prod), each with the variables
#     AZURE_CLIENT_ID, AZURE_TENANT_ID, and AZURE_SUBSCRIPTION_ID.
name: Deploy AI Gateway

on:
  workflow_dispatch:
    inputs:
      environment:
        description: Target environment
        type: choice
        options:
          - dev
          - prod
        default: dev
  push:
    branches:
      - main
    paths:
      - infra/**

permissions:
  id-token: write
  contents: read

concurrency:
  group: deploy-ai-gateway-${{ inputs.environment || 'dev' }}
  cancel-in-progress: false

jobs:
  deploy:
    runs-on: ubuntu-latest
    environment: ${{ inputs.environment || 'dev' }}
    env:
      DEPLOYMENT_LOCATION: eastus2
      PARAMETERS_FILE: ${{ (inputs.environment || 'dev') == 'prod' && 'infra/prod.bicepparam' || 'infra/main.bicepparam' }}
    steps:
      - name: Check out the repository
        uses: actions/checkout@v4

      - name: Sign in to Azure with OpenID Connect
        uses: azure/login@v2
        with:
          client-id: ${{ vars.AZURE_CLIENT_ID }}
          tenant-id: ${{ vars.AZURE_TENANT_ID }}
          subscription-id: ${{ vars.AZURE_SUBSCRIPTION_ID }}

      - name: Install the aigateway CLI extension
        run: curl -fsSL https://aka.ms/aigateway-cli-install | sh

      - name: Preview changes
        run: |
          az deployment sub what-if \
            --name "aigw-${{ github.run_id }}" \
            --location "$DEPLOYMENT_LOCATION" \
            --template-file infra/main.bicep \
            --parameters "$PARAMETERS_FILE"

      - name: Deploy
        id: deploy
        run: |
          OUTPUTS="$(az deployment sub create \
            --name "aigw-${{ github.run_id }}" \
            --location "$DEPLOYMENT_LOCATION" \
            --template-file infra/main.bicep \
            --parameters "$PARAMETERS_FILE" \
            --query properties.outputs --output json)"
          # Deployment output names can come back with a different first-letter casing, so match case-insensitively.
          out() { jq -r --arg k "$1" 'to_entries[] | select((.key | ascii_upcase) == $k) | .value.value' <<<"$OUTPUTS"; }
          {
            echo "resource-group=$(out AI_GATEWAY_RESOURCE_GROUP)"
            echo "gateway=$(out AI_GATEWAY_NAME)"
            echo "gateway-url=$(out AI_GATEWAY_URL)"
            echo "alias=$(out AI_GATEWAY_MODEL)"
          } >> "$GITHUB_OUTPUT"

      - name: Connect telemetry
        env:
          RG: ${{ steps.deploy.outputs.resource-group }}
          GATEWAY: ${{ steps.deploy.outputs.gateway }}
        run: bash infra/scripts/connect-telemetry.sh

      - name: Apply gateway baseline policies
        env:
          RG: ${{ steps.deploy.outputs.resource-group }}
          GATEWAY: ${{ steps.deploy.outputs.gateway }}
        run: bash infra/scripts/apply-baselines.sh infra/baselines.json

      - name: Smoke test
        env:
          RG: ${{ steps.deploy.outputs.resource-group }}
          GATEWAY: ${{ steps.deploy.outputs.gateway }}
        run: bash infra/scripts/smoke-test.sh "${{ steps.deploy.outputs.alias }}" web-chat learn

      - name: Summary
        run: |
          {
            echo "### AI Gateway deployed"
            echo "- Environment: ${{ inputs.environment || 'dev' }}"
            echo "- Gateway: ${{ steps.deploy.outputs.gateway }}"
            echo "- Endpoint: ${{ steps.deploy.outputs.gateway-url }}"
            echo "- Alias: ${{ steps.deploy.outputs.alias }}"
          } >> "$GITHUB_STEP_SUMMARY"

On GitHub, open infra/deploy-workflow.yml.

A push to main that changes infra/ deploys dev. Prod deploys only when you start the workflow manually and a reviewer approves it.

1

Run it

Bash
# Start the workflow for the dev environment
gh workflow run deploy-ai-gateway.yml -f environment=dev -R "$GITHUB_REPO" &&
  echo "⏳ The Deploy AI Gateway workflow is queued for dev"

# Watch the run's jobs and steps until it finishes
echo "⏳ Watching the run (Ctrl+C stops watching, not the run):"
gh run watch -R "$GITHUB_REPO"

Or, in GitHub, select ActionsDeploy AI GatewayRun workflow. The pipeline deploys to the same resource groups you created from your machine, because the resource token depends only on the subscription, the environment name, and the location. On the Bicep track, the pipeline doesn't set AZURE_PRINCIPAL_ID, so it doesn't declare your developer roles, and incremental deployments leave the ones you already have in place. On the Terraform track, the pipeline reads and updates the state you moved to the container, so its plan starts from what you deployed.

Checkpoint

The run's Smoke test step prints Smoke test passed., and the run summary lists the gateway, its endpoint, and the alias chat.

Verify

Prove that both runtime paths work, that the policy layers are in place, and that the identity boundaries hold.

1

Run the smoke test

The same script runs at the end of the pipeline. It calls the alias with the web-chat key, then runs the MCP handshake and tools/list on the learn MCP server. New role assignments can take a few minutes to apply, so it retries the model call for about a minute.

# Call the alias with the web-chat key, then run the MCP handshake and tools/list on the learn MCP server
echo "📊 Smoke test of the $AI_GATEWAY_MODEL alias and the learn MCP server:"
./infra/scripts/smoke-test.sh "$AI_GATEWAY_MODEL" web-chat learn

On GitHub, open infra/scripts/smoke-test.sh or infra/scripts/smoke-test.ps1.

Expected output (the tool list can change as the Learn server evolves):

Text
📊 Smoke test of the chat alias and the learn MCP server:
Gateway: https://aigw-abcde-k3v7qz2m9xw4p.eastus2-01.ai.gateway.azure.com
Model  : HTTP 200, model=<model>, reply=ok, total_tokens=<n>
Tools  : HTTP 200, 3 tools on learn
         learn_microsoft_docs_search
         learn_microsoft_code_sample_search
         learn_microsoft_docs_fetch
Smoke test passed.

That one model call passed every layer: the Gateway scope request rate and cost limits, then the model's token limit and content safety. The gateway called Foundry with its managed identity.

2

Review the policy layers in the portal

Select Policies. Request rate limit and Cost limit show Gateway in the Scope column. Token rate limit and Content safety show the backend model instead. Select Models to see that model and the alias chat, and MCP servers to see learn.

3

Call the alias with your Microsoft Entra identity

The dev parameters gave you AI Gateway Models User on the alias, so you can call it without a key:

# Get a Microsoft Entra token for AI Gateway, for your own identity
export AI_GATEWAY_TOKEN="$(az account get-access-token \
  --scope https://ai.gateway.azure.com/Data.All --query accessToken -o tsv)"
echo "👤 Token acquired: in AI_GATEWAY_TOKEN (${#AI_GATEWAY_TOKEN} characters, not shown)"

# Call the alias with the token instead of a key: your AI Gateway Models User role on the alias lets you in
curl -sS "${AI_GATEWAY_URL%/}/default/models/openai/v1/chat/completions" \
  -H "Authorization: Bearer $AI_GATEWAY_TOKEN" \
  -H "Content-Type: application/json" \
  -d "{\"model\":\"$AI_GATEWAY_MODEL\",\"messages\":[{\"role\":\"user\",\"content\":\"Say hello in five words.\"}]}" \
  | jq -r 'if .choices then "🤖 \(.choices[0].message.content)" else "🚨 \(.error.message // .)" end'

The reply is a short greeting. Your Entra identity has its own counters, separate from the web-chat key's.

4

Check the backend identity boundary

Confirm that the Foundry account rejects keys and that only the gateway's identity has Foundry User on it:

# Check that the Foundry account rejects keys: disableLocalAuth is true when key authentication is off
echo "🔑 disableLocalAuth on the Foundry account:"
az cognitiveservices account show -g "$FOUNDRY_RG" -n "$FOUNDRY_NAME" --query properties.disableLocalAuth

# List the roles that the gateway's managed identity has on the Foundry account
echo "🔐 Roles of the gateway's identity on the Foundry account:"
az role assignment list --scope "$FOUNDRY_ID" --assignee "$AI_GATEWAY_PRINCIPAL_ID" \
  --query "[].roleDefinitionName" -o tsv

Expected output:

Text
🔑 disableLocalAuth on the Foundry account:
true
🔐 Roles of the gateway's identity on the Foundry account:
Foundry User
5

Watch the traffic in Monitoring

Select Monitoring. After a few minutes, Overview shows the smoke test's calls under the web-chat key and your Entra call under your identity, and Traces shows each request with its policy spans. Run the smoke test a few more times if the charts are still empty.

Checkpoint

The smoke test passed, Policies shows two Gateway scope policies plus the model's own, a keyless call with your Entra token worked, the Foundry account has disableLocalAuth set to true, and Monitoring shows the traffic.

Go further

Clean up

The Foundry model deployment, Log Analytics, and Application Insights are billed normally, so delete what you deployed.

Choose your track
1

Delete the resource groups

Delete both resource groups. Each command waits until the group is gone, which can take several minutes:

# Delete the gateway's resource group, with the gateway, its keys, and the monitoring resources
az group delete -n "$RG" --yes &&
  echo "🚮 Resource group $RG is deleted"

# Delete the resource group with the Foundry account and its model deployment
az group delete -n "$FOUNDRY_RG" --yes &&
  echo "🚮 Resource group $FOUNDRY_RG is deleted"
2

Purge the Foundry account

Deleted Foundry accounts keep soft-delete records that hold their names. To reuse the name right away, purge it. Purging is permanent.

# Purge the deleted Foundry account, so that you can reuse its name right away
az cognitiveservices account purge -g "$FOUNDRY_RG" -n "$FOUNDRY_NAME" -l "$LOCATION" &&
  echo "🚮 Foundry account $FOUNDRY_NAME is purged"

If you set up GitHub Actions, remove the deployment identity's role assignments and the app registration, then delete the dev and prod environments in SettingsEnvironments:

# Remove the deployment identity's role assignments on the subscription
az role assignment delete --assignee "$SP_ID" --scope "/subscriptions/$SUBSCRIPTION_ID" &&
  echo "🚮 The deployment identity's role assignments are deleted"

# Delete the app registration, with its federated credentials
az ad app delete --id "$APP_ID" &&
  echo "🚮 App registration github-aigw-deploy is deleted"

If you deployed prod, repeat the clean-up with the aigw-prod deployment's outputs. On the Terraform track, run terraform init -reconfigure with the same -backend-config options and key=prod.tfstate, then terraform destroy -var-file prod.tfvars. Delete the state's storage account last, after you destroy every environment whose state it holds.

Next steps

Finished the lab?

Mark it complete to track your progress across paths. It's saved only in this browser.