Azure AI Gateway Labs
Documentation Open portal
Agents · Lab 17

Build on your gateway with coding agents

Install the AI Gateway plugin in GitHub Copilot CLI, Claude Code, Cursor, or Gemini CLI, let your coding agent discover the gateway's models and MCP tools and generate working client code, then migrate AI assets from Azure API Management.

Intermediate 45 min CodeCLIBicepTerraformNotebook Updated
Notebook Source Edit

What you'll do

  • Install the aigateway plugin in your coding agent
  • Discover a gateway's model aliases and MCP tool servers from its Azure resource ID
  • Generate and run app code that calls the gateway with a runtime key
  • Review generated code for the right endpoint, alias, and secret handling
  • Migrate model APIs and MCP servers from Azure API Management, directly or as Bicep or Terraform

Before you start

Tracks: Code, Azure CLI, Bicep, Terraform, Notebook

Your platform team publishes models and MCP tools behind AI Gateway, and product developers want to use them without reading gateway settings or passing keys around in chat. The AI Gateway plugin teaches coding agents such as GitHub Copilot CLI, Claude Code, Cursor, and Gemini CLI what your gateway offers. It then generates code that uses the right endpoint, alias, and auth header. Its developer commands only read from the gateway, so developers build on top of your governance instead of around it. Administrators use the same plugin to create and manage gateways, and to migrate the model APIs and MCP servers that they run on Azure API Management.

Before you begin

One-time setupThe Setup guide tasks that this lab uses. Mark each one done once, and the labs stop listing it.

Set these variables in the shell that you'll start your coding agent from. Create a key for this app so you can revoke it on its own later.

# Name your gateway and the alias that the generated app calls
export RG="<resource-group>"
export GATEWAY="<gateway>"
export AI_GATEWAY_MODEL="<model-name>"   # the alias from lab 02
echo "📝 Gateway $GATEWAY in $RG, alias $AI_GATEWAY_MODEL"

# Read the gateway's resource ID, which you pass to the plugin, and its runtime URL, which the app calls
export GATEWAY_ID="$(az aigateway show -g "$RG" -n "$GATEWAY" --query id -o tsv)"
export AI_GATEWAY_URL="https://$(az aigateway show -g "$RG" -n "$GATEWAY" --query properties.frontend.defaultHostname -o tsv)"
echo "🌐 Runtime URL: $AI_GATEWAY_URL"

# Create an API key for this app, so that you can revoke it on its own later
az aigateway api-key create -g "$RG" --gateway-name "$GATEWAY" -n app-coding-agent -o none &&
  echo "🔑 API key app-coding-agent is ready"

# Read the key into an environment variable, so that it never shows on screen
AI_GATEWAY_API_KEY="$(az aigateway api-key list-secrets -g "$RG" --gateway-name "$GATEWAY" -n app-coding-agent \
  --query primaryKey -o tsv)" && export AI_GATEWAY_API_KEY
echo "🔑 Key: in AI_GATEWAY_API_KEY (${#AI_GATEWAY_API_KEY} characters, not shown)"

Then print the gateway's Azure resource ID. You pass it to the plugin.

# Print the gateway's resource ID, to copy into the plugin's commands
echo "🆔 Gateway resource ID: $GATEWAY_ID"

The ID looks like /subscriptions/<subscription-id>/resourceGroups/<resource-group>/providers/Microsoft.ApiManagement/aigateways/<gateway>.

How it works

The aigateway plugin bundles commands and skills for two personas. Developers use discover and build to use a gateway's models and tools in their apps. Administrators use create and manage to run the gateway itself. The skills hold the know-how: which endpoints to call, which model name the gateway accepts, and how to send the credential. The commands run a skill for one task. This lab uses the developer commands, then migrate.

Discovery goes through Azure Resource Manager, which is why the plugin needs the resource ID and your Azure sign-in. Your app then calls the gateway's runtime URL with a runtime key or a Microsoft Entra token, like any other client. The developer commands are read-only against the gateway: they never create, update, or delete gateways, models, tools, or keys. The administrator commands change the gateway through the az aigateway CLI, and ask you to confirm before they delete anything, change access, or read a secret.

PieceInvoked asWhat it does
Skill use-ai-gateway/aigateway:use-ai-gatewayThe full discover, select, connect, and verify workflow. discover and build use it.
Command discover/aigateway:discover <resource ID>Lists the gateway's model aliases and MCP tool servers, with each endpoint, operation, and auth requirement, and key metadata when you ask. Stops there.
Command build/aigateway:build <resource ID> <description>Discovers what's missing, integrates the assets into your project with a credential you already have, then verifies one request.
Commands create and manage/aigateway:create, /aigateway:manageCreate a gateway, then configure and operate its models, tools, policies, and keys with the skill manage-ai-gateway. Both take the gateway name and resource group, then a region or the operation.
Command migrate/aigateway:migrate <apim-name> <resource-group> [target-gateway]Moves model APIs, MCP servers, and A2A agents from an API Management instance to AI Gateway in four steps: inventory, plan, adapt the plan with you, and provision it phase by phase. It runs the migrate-apim tool that the plugin bundles, with the skill migrate-apim.
Command feedback/aigateway:feedbackAnonymizes a session summary into a local file that you can share as product feedback.

What the generated code relies on:

ItemValue
Model base URL (OpenAI-compatible)<gateway URL>/default/models/openai/v1
Model operationOne that the alias publishes: Chat Completions, Responses, Anthropic Messages, or embeddings
MCP endpoint<gateway URL>/default/toolservers/<server-name>/mcp
Auth headerAuthorization: Bearer <credential>, a runtime key or a Microsoft Entra token, for both models and MCP tools
CredentialA Microsoft Entra token or a key that you already have. A key comes from the AI_GATEWAY_API_KEY environment variable
Model nameThe alias as the gateway publishes it, for example gpt-5-mini
ProjectYour existing project, in its language and with its dependencies. A standalone agent only if you ask for one

Install the plugin

1

Create a project folder

Start in an empty folder with Git, so you can check later that no secret gets committed.

# Create the project folder, move into it, and start a Git repository quietly
mkdir ticket-summarizer && cd ticket-summarizer && git init -q &&
  echo "📂 Project folder with Git: $(pwd)"
2

Add the plugin to your agent

The plugin lives in the Azure/aigateway repository and follows each agent's plugin format. In GitHub Copilot CLI and Claude Code, start a session in the folder (copilot or claude) and run the slash commands. In Gemini CLI, run the command in your shell, then start gemini in the folder. In Cursor, use the dashboard, then open the folder.

/plugin marketplace add Azure/aigateway
/plugin install aigateway@azure

The marketplace is named azure and the plugin aigateway. GitHub Copilot CLI reports Plugin "aigateway" installed successfully. Installed 4 skills. Earlier versions of the plugin were named ai-gateway. If a guide still shows /plugin install ai-gateway@azure, use aigateway@azure instead.

3

Check that the commands are available

In your agent, run the discover command without a resource ID:

Text
/aigateway:discover

The plugin lists the gateways that your Azure sign-in can read in the current subscription and asks which one to use. It only reads. That's how you know it's loaded. Answer with your gateway's name or the ID you printed, or continue with the next step.

Checkpoint

/aigateway:discover is recognized and asks which gateway to use.

Discover what the gateway offers

1

Run discover with your resource ID

Paste the gateway ID that you printed in Before you begin. Discover lists a tool server's tools only with a credential that you already have, so tell it to use the key you created for this app:

Text
/aigateway:discover /subscriptions/<subscription-id>/resourceGroups/<resource-group>/providers/Microsoft.ApiManagement/aigateways/<gateway> For the MCP handshake, use the key in the AI_GATEWAY_API_KEY environment variable.

When your agent asks to run az, curl, or python3 commands, approve the ones that read: az aigateway and az rest GET requests to management.azure.com, and MCP calls to your gateway URL, which the plugin may send from a short Python script. Deny any PUT, PATCH, or DELETE. Discover never needs them.

2

Read the summary

The plugin reads the gateway's runtime URL and lists each model alias with the operations it publishes, such as Chat Completions at /default/models/openai/v1/chat/completions and Responses at /default/models/openai/v1/responses. It lists the MCP tool servers with their endpoints. With your key, it runs an MCP handshake against each tool server and lists its tools, such as learn_microsoft_docs_search, without calling them. Ask follow-up questions in plain language, for example:

Text
Which model fits short summaries? Do any tool servers search Microsoft Learn? Which API keys does the gateway have?

For the last question, it lists the gateway's API keys from their metadata, without their values.

Discover doesn't write code, read key secrets, call models or tools, or change the gateway. If you don't name a credential, it still lists the aliases and tool servers, but not the tools. Stop here, or move on to build.

Checkpoint

The summary lists the alias you published in lab 02 and, if you did lab 04, the learn tool server and its tools.

Build an app with the gateway

1

Describe what to build

Run build with the same resource ID and a short description. build uses a credential that you already have, so name the environment variable that holds the key you created for this app. Otherwise it asks which credential to use:

Text
/aigateway:build /subscriptions/<subscription-id>/resourceGroups/<resource-group>/providers/Microsoft.ApiManagement/aigateways/<gateway> a Python command-line script that summarizes a support ticket from a text file. Use the <model-name> model and read the key from the AI_GATEWAY_API_KEY environment variable.

Other descriptions to try:

You wantDescribe it as
A model call in an app you already haveadd a /summarize endpoint to my FastAPI app that calls the gateway model
Tools in your own MCP clientconnect my app to the learn MCP tools
A ready-to-run agenta standalone agent that answers Azure questions with the learn tools
2

Answer the plugin's questions

build reuses what you gave it, discovers only what's missing, and asks what you're building, if your description didn't say. It works in the project that's in the folder and follows its language, framework, and dependencies:

  • Call a model: code for an operation that the alias publishes, such as Chat Completions or Responses.
  • Connect MCP tools: it lists the server's tools first, then configures the ones you choose by their exact names in the MCP client that your project uses. It asks before it calls a tool.
  • Build a standalone agent: only when you ask for one, as in the last example above.

build sends the credential as an Authorization: Bearer header and keeps it where it is, such as in AI_GATEWAY_API_KEY. It never reads, creates, or rotates keys, so it writes no key to a file: an .env.example holds only an empty placeholder. Then it verifies the code with one request. If that request costs tokens and your description didn't approve it, build asks you first. If you decline, it tells you that the request wasn't verified.

3

Review the generated code

Read the code before you run it again or commit it. Check that:

  • The base URL is your gateway URL followed by /default/models/openai/v1, or /default/toolservers/<name>/mcp for tools. Without /default/, calls return 404.
  • The model value is the alias that discover listed, so you can retarget models later without a code change, as explained in Publish models from any provider behind stable aliases.
  • The key comes from AI_GATEWAY_API_KEY and goes in the Authorization: Bearer header. No key value appears in code, README.md, or config files, and .env.example holds only an empty placeholder.
  • If the project has a .env file, .gitignore lists it.
  • The app calls only the gateway's runtime URL, never management.azure.com.
  • For MCP tools: the configuration lists the selected tools by name instead of "*", and tool calls wait for your decision instead of approving all.
4

Run the app

build reported how it checked the code and the command that runs it. Run the app yourself from the shell where you set AI_GATEWAY_API_KEY, using that command or the generated README.md. If the project has a requirements.txt, install its packages in a virtual environment first:

# Create a virtual environment in .venv and activate it, so that the packages stay out of your global Python
python3 -m venv .venv && source .venv/bin/activate &&
  echo "📦 Virtual environment .venv is active"

# Install the packages that the generated app lists in requirements.txt
pip install -r requirements.txt &&
  echo "📦 The packages in requirements.txt are installed"

Then start the entry point, for example python summarize_ticket.py ticket.txt. A script that uses only the Python standard library has no requirements.txt, so run it directly.

Checkpoint

The generated app prints a model response that came through your gateway.

Migrate from API Management

If you already serve models and MCP servers through Azure API Management, /aigateway:migrate moves them to AI Gateway. It runs the migrate-apim tool that the plugin bundles, in four steps. Each step writes a file in the folder that you work in, so you can review it, stop, and pick up where you left off.

StepWhat the agent runsWhat it writes
Inventorymigrate-apim collect, discover-tools, and summary read the API Management instance: facts only, no verdictsinventory.json
Planmigrate-apim plan matches each asset against what the installed az aigateway can createplan.json
AdaptYou settle each item that doesn't map one to one, and the agent records your decisions in the planplan.json
Provisionmigrate-apim apply builds the gateway one phase at a time, or the agent writes Bicep or Terraform from the planjournal.json, or a template

The migration is additive: it builds a new gateway and changes nothing in API Management. Callers keep using API Management until you move them, and retiring anything there stays your decision.

1

Deploy a sample API Management instance

Skip this step if you migrate your own instance. Set MIG_RG to its resource group, APIM to its name, and FOUNDRY_ID to the resource ID of the Foundry account that it calls.

Otherwise, deploy a sample that serves AI the way many API Management instances do: an Azure OpenAI API in front of a new Foundry account with gpt-5-mini, a token limit for each subscription, a header that names the calling app, a product with one subscription for a ticket app, and an MCP server that passes through to the Microsoft Learn MCP server. It takes about 10 minutes. API Management BasicV2 bills by the hour until you delete it.

# Name the resource group, the region, and the sample's resources, with a random suffix
export MIG_RG="rg-aigw-lab17mig"
export MIG_LOCATION="swedencentral"
export SUFFIX="$(openssl rand -hex 3)"
export APIM="apim-contoso-$SUFFIX"
export FOUNDRY="aigwsrc$SUFFIX"
echo "📝 API Management $APIM and Foundry account $FOUNDRY, in $MIG_RG"

# Create the resource group
az group create -n "$MIG_RG" -l "$MIG_LOCATION" -o none && echo "📂 Resource group $MIG_RG is ready"

# Deploy API Management, the Foundry account, the model API, and the MCP server. It takes about 10 minutes
az deployment group create -g "$MIG_RG" -n apim-source \
  -f ~/aigateway/labs/17-coding-agent-plugin/infra/apim-source.bicep \
  -p apimName="$APIM" foundryName="$FOUNDRY" -o none &&
  echo "🚀 API Management $APIM is deployed"

# Read the Foundry account's resource ID, which the migrated gateway calls
export FOUNDRY_ID="$(az cognitiveservices account show -g "$MIG_RG" -n "$FOUNDRY" --query id -o tsv)"
echo "🆔 Foundry account: $FOUNDRY_ID"

To use another region, pick one where API Management BasicV2, the gpt-5-mini model, and AI Gateway are all available, as in Find the available regions. On GitHub, open infra/apim-source.bicep to see the policies that the migration has to carry over.

2

Start the migration in your agent

Create a folder for the migration files, then start your coding agent in it.

# Create the folder that holds inventory.json, plan.json, and journal.json, and move into it
mkdir -p ~/apim-migration && cd ~/apim-migration && echo "📂 Migration folder: $(pwd)"

Ask your agent for the first two steps only. Replace <apim-name> and <resource-group> with the values of APIM and MIG_RG.

Text
/aigateway:migrate <apim-name> <resource-group>
Work in the current folder. Do only the inventory and the review: collect the inventory, show the summary,
discover the MCP tools, and propose the plan. Then present every item that needs a decision, with its options
and your recommendation, and list the items that are ready to migrate and the prerequisites. Don't record
decisions and don't run apply yet.

The agent says which copy of the tool it runs, usually the plugin's own copy, and asks you before it calls an MCP server. It writes inventory.json and plan.json in the folder, then presents the items to decide.

The next steps show each command that the agent runs and what it prints, so you can check its work or run the steps yourself. To run them, define a shortcut to the same tool in your clone of the repo. It's a Python package with no dependencies.

# Run the migrate-apim tool in place from your clone of the repo, as the plugin runs its own copy
migrate-apim() { PYTHONPATH=~/aigateway/tools/migrate-apim python3 -m migrate_apim.cli "$@"; }
migrate-apim --version && echo "🔧 migrate-apim is ready"
3

Inventory the AI assets

collect reads the instance through Azure Resource Manager and writes inventory.json: the backends, the APIs and their operations, the policies, the products and subscriptions, and the Foundry deployments behind them. It records how to retrieve each secret, never its value. --operation-policies also reads each operation's policy, at one read per operation.

# Inventory the AI assets of the API Management instance, with every operation's policy
migrate-apim collect --apim "$APIM" -g "$MIG_RG" --operation-policies -o inventory.json

It prints a line for each read, a summary, and Wrote inventory.json.

The inventory can't know which tools a passthrough MCP server exposes until something asks the server. Asking calls the server, maybe with a subscription key, so discover-tools calls nothing until you authorize the server.

# See which servers need your authorization: nothing is called yet
migrate-apim discover-tools --inventory inventory.json

# Authorize the call to learn-mcp, and record its tools in inventory.json
migrate-apim discover-tools --inventory inventory.json --allow-secret-use mcp:learn-mcp

The first run calls nothing:

Text
  needsAuthorization mcp:learn-mcp via none
                     Listing tools calls the server; not authorized.

Nothing was called. Listing tools calls the server and may use a subscription key, so it needs explicit authorization:
  --allow-secret-use <assetId>   (or 'all')

The second run lists the tools through the API Management gateway, which is what callers see:

Text
  discovered         mcp:learn-mcp via gateway
                     microsoft_docs_search, microsoft_code_sample_search, microsoft_docs_fetch
                     3 tools

Updated inventory.json (1 server(s) discovered)

Print the summary of the inventory:

# Print a readable summary of inventory.json
migrate-apim summary inventory.json
Text
APIM AI inventory: apim-contoso-<suffix>
  /subscriptions/<subscription-id>/resourceGroups/rg-aigw-lab17mig/providers/Microsoft.ApiManagement/service/apim-contoso-<suffix>
  sku BasicV2, region Sweden Central, vnet None, identity SystemAssigned
  collected <time> in 30 ARM reads

Assets
  1 model providers
      modelProvider:foundry-ai-endpoint [foundry] endpoint=https://aigwsrc<suffix>.cognitiveservices.azure.com/openai auth=managedIdentity models=1 refs=0 apis/1 policies
  1 model APIs
      azure-openai [inferenceProxy, confidence=high] provider=foundry auth=managedIdentity models=1
  1 MCP servers
      learn-mcp [passthrough, confidence=high] tools=3 upstream=1
  0 A2A agents
  0 APIs excluded as non-AI

Secrets
  3 references, 3 retrievable automatically, 0 needing manual re-provisioning
      apimSubscriptionKey: 2
      azureManagedIdentity: 1

Policy surface
  2 documents, 9 elements, 2 distinct policy expressions
  top families: composition (4), routing (2), other (1), tokenLimit (1), transform (1)

The inventory found the three assets to migrate: the Foundry backend as a model provider, the Azure OpenAI API as a model API that signs in to Foundry with API Management's managed identity, and the MCP server with its three tools. It also found the secrets that callers use, here two subscription keys, and the policies that the plan must translate.

4

Review the migration strategy for each asset

plan matches every asset against what the installed az aigateway can create, and writes plan.json. Each asset becomes one or more items: an item is ready when it maps one to one, and needs a decision when it doesn't.

# Propose a migration plan from the inventory
migrate-apim plan --inventory inventory.json -o plan.json
Text
note: this plan says nothing about where it deploys, and `apply` refuses without it.
      Re-run with --target /subscriptions/<sub>/resourceGroups/<rg>/providers/Microsoft.ApiManagement/aigateways/<gateway>
Migration plan: apim-contoso-<suffix>
  10 items against CLI 1.0.0b31 (declared, not probed)
  6 needs-decision, 4 ready

Needs a decision (6, 3 blocking)
  accept-loss: 1
  name: 1
  reduce-surface: 1
  scope: 1
  translate: 2
    ! [reduce-surface] model:azure-openai -> proposal=split of split/inference-only/retain-in-apim/block
      [name] model:azure-openai -> proposal=use-model-names of use-model-names/override
      [translate] policy:llm-token-limit -> proposal=identity of identity/ipAddress/identity+ipAddress/pooled
    ! [accept-loss] policy:set-header -> proposal=accept of accept/compensate/retain-in-apim/block
    ! [scope] access:subscriptions -> proposal=entra-rbac of gateway-key-per-consumer/entra-rbac/retain-in-apim
      [translate] telemetry:destination -> proposal=reuse-appinsights of reuse-appinsights/new-appinsights/gateway-metrics-only/retain-in-apim

Ready to execute (4)
    identity.assign: identity [gatewayIdentity]
    provider.create: modelProvider:foundry-ai-endpoint [registration]
    model.create: model:azure-openai [model:gpt-5-mini]
    mcp.create: mcp:learn-mcp [registration]

Manual prerequisites (4)
  [azure-administrator] Grant the gateway identity data-plane access on each Foundry account
  [azure-administrator] Enable the OTLP endpoint on the Application Insights resource APIM reports to
  [azure-administrator] Assign callers AI Gateway Models User / Tools User at the right scope
  [migrate-apim] Preflight the target name against Microsoft.ApiManagement/service

Wrote plan.json (28,110 bytes)

A ! marks a blocking item: nothing gets built until you settle it. Each item in plan.json has a rationale, its options with their consequences, and a proposal. Your agent presents them with its recommendation. This lab settles them as follows:

ItemWhy it needs a decisionOptionsThis lab's choice
model:azure-openai, published surface2 of the 3 operations map to the gateway's routes: image generation has no targetsplit, inference-only, retain-in-apim, blocksplit: chat completions and embeddings move, image generation stays on API Management
model:azure-openai, alias namesApps send the alias as model, and API Management has noneuse-model-names, overrideuse-model-names: apps keep sending gpt-5-mini
policy:llm-token-limitAPI Management counts tokens by subscription, and the gateway by identity, IP address, both, or one pool. estimate-prompt-tokens and remaining-tokens-header-name have no equivalentidentity, ipAddress, identity+ipAddress, pooledidentity: each key or Entra identity gets 5,000 tokens a minute
policy:set-headerThe gateway doesn't rewrite requests, so the x-caller-app header is lostaccept, compensate, retain-in-apim, blockaccept: the sample's backend doesn't read the header
access:subscriptionsGateway keys are gateway-wide, and no key value can be imported, so every caller gets a new credentialgateway-key-per-consumer, entra-rbac, retain-in-apimgateway-key-per-consumer: a key named after the ticket-app subscription
telemetry:destinationAPI Management reports to Application Insights, and the gateway exports OpenTelemetryreuse-appinsights, new-appinsights, gateway-metrics-only, retain-in-apimgateway-metrics-only: the gateway's built-in metrics, with nothing to set up

Your agent may recommend otherwise, and for production it should. When this lab was tested, the agent recommended entra-rbac, so that callers sign in instead of holding keys, compensate or retain-in-apim for the header, and new-appinsights, so that no shared resource changes. The lab picks keys and built-in metrics to stay self-contained: Go keyless with Microsoft Entra ID and RBAC and Attribute tokens and cost with OpenTelemetry cover the other options.

Tell your agent your decisions, and it records them. It sets only each item's status to adapted and its decision to the option you chose. This snippet records the same decisions for the sample:

# Record a decision for each item that needs one, keyed by the aspect of the source that it covers
python3 - <<'EOF'
import json
decisions = {
    "publishedSurface": "split",
    "aliasNames": "use-model-names",
    "llm-token-limit": "identity",
    "set-header": "accept",
    "subscriptions": "gateway-key-per-consumer",
    "destination": "gateway-metrics-only",
}
with open("plan.json", encoding="utf-8") as f:
    plan = json.load(f)
for item in plan["items"]:
    choice = decisions.get(item.get("source", {}).get("aspect"))
    if item.get("status") == "needs-decision" and choice:
        item["status"], item["decision"] = "adapted", choice
        print(f"📝 {item['id']}: {choice}")
with open("plan.json", "w", encoding="utf-8") as f:
    json.dump(plan, f, indent=2)
EOF

It prints the six items that it settled:

Text
📝 item:model/azure-openai/surface: split
📝 item:model/azure-openai/aliasNames: use-model-names
📝 item:policy/52c23f672ef3: identity
📝 item:policy/800b18024fa2: accept
📝 item:access/subscriptions: gateway-key-per-consumer
📝 item:telemetry/destination: gateway-metrics-only

The two policy items are named after a hash of the policy, so their IDs differ in your plan.

5

Identify the assets that are ready to migrate

Run plan again on the same file. It keeps your decisions, turns them into steps, and recomputes what's ready.

# Regenerate the plan in place, with your decisions
migrate-apim plan --inventory inventory.json -o plan.json
Text
note: this plan says nothing about where it deploys, and `apply` refuses without it.
      Re-run with --target /subscriptions/<sub>/resourceGroups/<rg>/providers/Microsoft.ApiManagement/aigateways/<gateway>
Migration plan: apim-contoso-<suffix>
  10 items against CLI 1.0.0b31 (declared, not probed)
  6 adapted, 4 ready

Needs a decision (0, 0 blocking)

Ready to execute (4)
    identity.assign: identity [gatewayIdentity]
    provider.create: modelProvider:foundry-ai-endpoint [registration]
    model.create: model:azure-openai [model:gpt-5-mini]
    mcp.create: mcp:learn-mcp [registration]

Manual prerequisites (3)
  [azure-administrator] Grant the gateway identity data-plane access on each Foundry account
  [azure-administrator] Assign callers AI Gateway Models User / Tools User at the right scope
  [migrate-apim] Preflight the target name against Microsoft.ApiManagement/service

Wrote plan.json (30,035 bytes)

Nothing needs a decision anymore, so every asset is ready to migrate:

  • Ready items map one to one: the gateway's identity, the Foundry provider, the gpt-5-mini model, and the learn-mcp MCP server.
  • Adapted items, which plan.json holds but the summary doesn't list, carry your decisions: the chat completions and embeddings routes under the alias gpt-5-mini, the token limit for each identity, and the ticket-app key. The header and the image generation operation stay behind, as decided.
  • Manual prerequisites are what only an administrator can do, outside the gateway. The telemetry prerequisite is gone, because no exporter needs it. The gateway's identity needs a role on the Foundry account, and the callers need a role on the gateway if they sign in with Entra ID.
Checkpoint

plan.json shows 6 adapted, 4 ready and Needs a decision (0, 0 blocking).

Build the migrated gateway

Choose how to build the gateway that the plan describes. Migrate directly runs migrate-apim apply, which creates it with az aigateway, one phase at a time, and keeps a journal so that you can resume. Bicep and Terraform turn the plan into infrastructure as code that you review, commit, and deploy with your pipelines.

Choose your track
1

Point the plan at a new gateway

apply refuses a plan that doesn't say where it deploys. Record the new gateway's resource ID in the plan. Later runs of plan keep it, so a resumed session doesn't have to repeat it.

# Name the new gateway, and build its resource ID in the migration's resource group
export MIG_GATEWAY="aigw-contoso-$SUFFIX"
export TARGET_ID="/subscriptions/$(az account show --query id -o tsv)/resourceGroups/$MIG_RG/providers/Microsoft.ApiManagement/aigateways/$MIG_GATEWAY"
echo "🆔 Target gateway: $TARGET_ID"

# Record the target in the plan, which adds the step that creates the gateway
migrate-apim plan --inventory inventory.json -o plan.json --target "$TARGET_ID"
Text
🆔 Target gateway: /subscriptions/<subscription-id>/resourceGroups/rg-aigw-lab17mig/providers/Microsoft.ApiManagement/aigateways/aigw-contoso-<suffix>
Migration plan: apim-contoso-<suffix>
  11 items against CLI 1.0.0b31 (declared, not probed)
  6 adapted, 5 ready

Needs a decision (0, 0 blocking)

Ready to execute (5)
    gateway.create: gateway [create]
    identity.assign: identity [gatewayIdentity]
    provider.create: modelProvider:foundry-ai-endpoint [registration]
    model.create: model:azure-openai [model:gpt-5-mini]
    mcp.create: mcp:learn-mcp [registration]

Manual prerequisites (3)
  [azure-administrator] Grant the gateway identity data-plane access on each Foundry account
  [azure-administrator] Assign callers AI Gateway Models User / Tools User at the right scope
  [migrate-apim] Preflight the target name against Microsoft.ApiManagement/service

Wrote plan.json (31,769 bytes)

The plan now has 11 items: gateway.create comes first.

2

Preflight the migration

The preflight checks the plan and the target, lists the prerequisites, and prints the phases, without writing anything. Define a second shortcut for the phases too.

# Run one phase of the plan: a rehearsal that prints the commands, or a write with --execute
migrate-phase() { migrate-apim apply --plan plan.json --inventory inventory.json --phase "$@"; }

# Check the plan, the target, and the prerequisites, and list the phases
migrate-apim apply --plan plan.json --inventory inventory.json --preflight
Text
Preflight:
  [ note] prerequisites: 3 step(s) only an administrator can do: prereq:foundry-rbac, prereq:caller-rbac, prereq:name-collision
          Acknowledge each with --acknowledge <id> once it is done. Acknowledgement is recorded and does not gate writing. Where the grant is readable it is checked as well -- see foundryRoleMissing -- but a gateway builds cleanly without any of them while failing to serve traffic.

Phases to run:
  gateway      pending    (1 steps)
  identity     pending    (1 steps)
  providers    pending    (1 steps)
  models       pending    (1 steps)
  aliases      pending    (1 steps)
  toolServers  pending    (1 steps)
  policies     pending    (1 steps)
  keys         pending    (1 steps)

Put these in your task list now, one entry per phase, and tick them from what a run
prints rather than from memory -- this checklist is rebuilt from the journal every
time, and a list kept alongside it drifts.

apply runs a phase as a rehearsal unless you add --execute, and refuses to execute a phase that you haven't rehearsed. Your agent rehearses each phase, shows you the commands, and asks before it executes, one phase at a time. Always name the phase with --phase: with migrate-apim 0.1.0, --next-phase goes back to the providers phase after models.

3

Create the gateway and its identity

# Rehearse the gateway phase: it prints the az command that it would run
migrate-phase gateway

# Create the gateway. It takes about two minutes
migrate-phase gateway --execute

# Rehearse, then give the gateway a system-assigned identity
migrate-phase identity && migrate-phase identity --execute

Each run first prints the preflight note and the phases, with > on the phase that runs. Then a rehearsal prints the command, and an execution reads the result back. This excerpt shows the rest of the output:

Text
Phase gateway -- rehearsal (pass --execute to write)
  planned          item:gateway/create
                   aigw-contoso-<suffix>: dry run; not executed
                   $ az aigateway create --subscription <subscription-id> -g rg-aigw-lab17mig -n aigw-contoso-<suffix> --location Sweden Central -o json

Rehearsed gateway. Nothing was written. Run the same phase with --execute to apply it.

Phase gateway -- EXECUTING
  verified         item:gateway/create
                   aigw-contoso-<suffix>: created and read back

Phase gateway complete. Next: identity.

Phase identity -- EXECUTING
  verified         item:identity/assign
                   created and read back

Phase identity complete. Next: providers.

The gateway is created in the region of API Management.

4

Publish the model

The providers phase of migrate-apim 0.1.0 fails with az aigateway 1.0.0b34: it copies API Management's backend URL, which ends in /openai, so inference would fail, and it sends no display name (#62). Create the provider and the model with az aigateway instead. --foundry-resource-id uses the account's root endpoint and grants the gateway's identity the Foundry User role on the account, the prerequisite that the plan lists first. Then let the plan check the model and publish the alias.

# Create the Foundry provider, and grant the gateway's identity access to the account
az aigateway model-provider create -g "$MIG_RG" --gateway-name "$MIG_GATEWAY" -n foundry-ai-endpoint \
  --foundry-resource-id "$FOUNDRY_ID" -o none && echo "🔌 Provider foundry-ai-endpoint is ready"

# Register the deployment that the model API served
az aigateway model create -g "$MIG_RG" --gateway-name "$MIG_GATEWAY" --provider-name foundry-ai-endpoint \
  -n gpt-5-mini --deployment gpt-5-mini --foundry-resource-id "$FOUNDRY_ID" -o none &&
  echo "🤖 Model gpt-5-mini is registered"

# Let the plan check the model, then publish the alias that apps call
migrate-phase models && migrate-phase models --execute
migrate-phase aliases && migrate-phase aliases --execute

The models phase finds the model that you registered, and the aliases phase publishes it under its name. An excerpt of the output:

Text
🔌 Provider foundry-ai-endpoint is ready
🤖 Model gpt-5-mini is registered

Phase models -- EXECUTING
  verified         item:model/azure-openai/model/gpt-5-mini
                   gpt-5-mini: already present with the desired state; apiFormat is not returned by a read and was not compared

Phase models complete. Next: providers.

Phase aliases -- rehearsal (pass --execute to write)
  planned          item:model/azure-openai/aliasNames
                   gpt-5-mini: dry run; not executed
                   $ az aigateway model-alias set --subscription <subscription-id> -g rg-aigw-lab17mig --gateway-name aigw-contoso-<suffix> -n gpt-5-mini --routing-name gpt-5-mini --backend-model {resource-id:/subscriptions/<subscription-id>/resourceGroups/rg-aigw-lab17mig/providers/Microsoft.ApiManagement/aigateways/aigw-contoso-<suffix>/workspaces/default/modelProviders/foundry-ai-endpoint/models/gpt-5-mini,priority:0,weight:100} -o json

Phase aliases -- EXECUTING
  verified         item:model/azure-openai/aliasNames
                   gpt-5-mini: created and read back

Phase aliases complete. Next: providers.
5

Federate the MCP server

# Rehearse, then create the MCP server
migrate-phase toolServers && migrate-phase toolServers --execute

# Check the MCP server on the gateway
az aigateway mcp show -g "$MIG_RG" --gateway-name "$MIG_GATEWAY" -n learn-mcp \
  --query properties.provisioningState -o tsv
Text
Phase toolServers -- EXECUTING
  failed           item:mcp/learn-mcp/registration
                   learn-mcp: the object exists but does not match the desired state

Stopped in toolServers. The source APIM is untouched; the gateway is partially built.
Succeeded

The phase stops with an error although the server exists: migrate-apim 0.1.0 compares it with the shape that an older az aigateway returned (#62). Succeeded confirms the server, so go on with the next phase.

6

Carry over the token limit and the keys

# Rehearse, then set the token limit on the model
migrate-phase policies && migrate-phase policies --execute

# Rehearse, then create a key for each API Management subscription
migrate-phase keys && migrate-phase keys --execute

An excerpt of the output:

Text
Phase policies -- EXECUTING
  verified         item:policy/52c23f672ef3
                   gpt-5-mini: created and read back; the source also set estimate-prompt-tokens=false, remaining-tokens-header-name=x-remaining-tokens, which the target has no equivalent for

Phase policies complete. Next: providers.

Phase keys -- EXECUTING
  verified         item:access/subscriptions
                   2/2 objects: master: already present with the desired state; ticket-app: created and read back

Phase keys complete. Next: providers.

The token limit is 5,000 tokens a minute for each identity, as decided. API Management's built-in all-access subscription maps to the master key that every gateway has, and the ticket-app subscription gets its own key.

7

Check the migration status

Acknowledge the Foundry role, which az aigateway granted when it created the provider. Your agent never acknowledges a prerequisite for you. Add --preflight: with migrate-apim 0.1.0, --acknowledge alone prints the status but doesn't record the acknowledgement in journal.json. Then print where the run is.

# Record in journal.json that the gateway's identity has its role on the Foundry account
migrate-apim apply --plan plan.json --inventory inventory.json --preflight --acknowledge prereq:foundry-rbac

# Print the state of each phase, from journal.json
migrate-apim apply --plan plan.json --inventory inventory.json --status

# Read the gateway's resource ID and runtime URL, which Verify uses
export MIG_GATEWAY_ID="$TARGET_ID"
export MIG_URL="https://$(az aigateway show -g "$MIG_RG" -n "$MIG_GATEWAY" --query properties.frontend.defaultHostname -o tsv)"
echo "🌐 Migrated gateway: $MIG_URL"

The --status run prints the preflight note, then:

Text
Migration run
  target: /subscriptions/<subscription-id>/resourceGroups/rg-aigw-lab17mig/providers/Microsoft.ApiManagement/aigateways/aigw-contoso-<suffix>
  steps: 8 in the plan, 1 failed, 6 verified
    gateway      complete   (1 steps)
    identity     complete   (1 steps)
    providers    pending    (1 steps)
    models       complete   (1 steps)
    aliases      complete   (1 steps)
    toolServers  stopped    (1 steps) (rehearsed)
    policies     complete   (1 steps)
    keys         complete   (1 steps)
  stuck:
    failed: item:mcp/learn-mcp/registration -- learn-mcp: the object exists but does not match the desired state
  acknowledged: prereq:foundry-rbac
🌐 Migrated gateway: https://aigw-conto-<hash>.swedencentral-01.ai.gateway.azure.com

The providers phase stays pending because you created the provider yourself, and toolServers shows the failed read-back. Every other phase is complete, and acknowledged lists the role. The other two prerequisites don't apply here: the sample's callers use a key, and every preflight checks the name. It stops the run if an API Management instance in the resource group has the gateway's name.

Checkpoint

The new gateway has the model gpt-5-mini with its alias, the MCP server learn-mcp, and the key ticket-app. API Management is unchanged and still serves the same calls.

Before you move callers from API Management to the gateway, tell their owners what changes:

  • The endpoint. Model calls move from https://<apim-name>.azure-api.net/openai/deployments/gpt-5-mini/... to <gateway-url>/default/models/openai/v1/..., with "model": "gpt-5-mini" in the body. MCP clients move to <gateway-url>/default/toolservers/learn-mcp/mcp, where each tool name starts with learn-mcp_.
  • The credential. No subscription key carries over: each caller gets its gateway key, or an Entra role.
  • What stays behind. Image generation stays on API Management. The token limit counts for each identity instead of each subscription, the x-remaining-tokens header is gone, and so is x-caller-app.

Verify

1

Confirm the key and model outside the generated code

If the app fails, this call tells you whether the problem is in the gateway setup or in the code.

# Call the model through the gateway with your key and alias, and print only the HTTP status code
echo "🤖 HTTP status of a model call through the gateway:"
curl -sS -o /dev/null -w "HTTP %{http_code}\n" "${AI_GATEWAY_URL%/}/default/models/openai/v1/chat/completions" \
  -H "Authorization: Bearer $AI_GATEWAY_API_KEY" -H "Content-Type: application/json" \
  -d "{\"model\":\"$AI_GATEWAY_MODEL\",\"messages\":[{\"role\":\"user\",\"content\":\"ping\"}]}"

Expect HTTP 200 (200 in PowerShell).

2

Check that no secret can be committed

Run this in the project folder. It checks that Git ignores .env, if the project has one, and that your key appears in no other file.

# If the project has a .env file, which holds a key, check that Git ignores it
if [ ! -e .env ]; then echo "✅ No .env file"
elif git check-ignore -q .env; then echo "✅ .env is ignored"
else echo "🚨 .env is not ignored"; fi

# Search every other file for your key, skipping .git, .venv, and node_modules, and list any file that holds it
if grep -rIlF --exclude=.env --exclude-dir=.git --exclude-dir=.venv --exclude-dir=node_modules -- "$AI_GATEWAY_API_KEY" .; then
  echo "🚨 The key appears in the files above"
else
  echo "✅ No key in project files"
fi

Expect ✅ No .env file when the code reads the key only from your environment, or ✅ .env is ignored, and ✅ No key in project files. If .env holds a different key, run the check again with that key in AI_GATEWAY_API_KEY.

3

Call the migrated model

If you migrated the sample, call gpt-5-mini through the new gateway with the ticket-app key, by the same model name that API Management served.

# Read the ticket-app key into a variable, so that it never shows on screen
MIG_KEY="$(az aigateway api-key list-secrets --ids "$MIG_GATEWAY_ID/apiKeys/ticket-app" --query primaryKey -o tsv)"
echo "🔑 Key: in MIG_KEY (${#MIG_KEY} characters, not shown)"

# Call the model through the migrated gateway, and print only the HTTP status code
echo "🤖 HTTP status of a model call through the migrated gateway:"
curl -sS -o /dev/null -w "HTTP %{http_code}\n" "$MIG_URL/default/models/openai/v1/chat/completions" \
  -H "Authorization: Bearer $MIG_KEY" -H "Content-Type: application/json" \
  -d '{"model":"gpt-5-mini","messages":[{"role":"user","content":"ping"}]}'

Expect HTTP 200 (200 in PowerShell).

4

List the migrated MCP tools

# List the tools of the migrated MCP server, and print their names
echo "🔌 Tools of learn-mcp on the migrated gateway:"
curl -sS "$MIG_URL/default/toolservers/learn-mcp/mcp" -H "Authorization: Bearer $MIG_KEY" \
  -H "Content-Type: application/json" -H "Accept: application/json, text/event-stream" \
  -d '{"jsonrpc":"2.0","id":1,"method":"tools/list"}' | grep -o '"name":"[^"]*"'

Expect the three tools that discover-tools found in API Management, each with the prefix learn-mcp_:

Text
"name":"learn-mcp_microsoft_docs_search"
"name":"learn-mcp_microsoft_code_sample_search"
"name":"learn-mcp_microsoft_docs_fetch"
Checkpoint

The direct call returns 200, the app works, and no key can reach Git. If you migrated the sample, the migrated gateway answers 200 and lists the three Microsoft Learn tools.

Go further

Use the gateway's tools in VS Code agent mode

VS Code reads MCP servers from .vscode/mcp.json. To give agent mode the tools that you federated in Federate MCP tools behind one endpoint, save this file as .vscode/mcp.json in your project. Replace <gateway-url> with your gateway URL (echo $AI_GATEWAY_URL) and learn with any tool server name that discover listed. To copy a tool server's exact MCP URL, run az aigateway mcp show -g "$RG" --gateway-name "$GATEWAY" -n learn --query "properties.gatewayEndpoints[0].url" -o tsv.

infra/mcp.jsonGitHub
{
  "inputs": [
    {
      "type": "promptString",
      "id": "aigw-key",
      "description": "AI Gateway runtime key (Access > API keys)",
      "password": true
    }
  ],
  "servers": {
    "ai-gateway-learn": {
      "type": "http",
      "url": "<gateway-url>/default/toolservers/learn/mcp",
      "headers": {
        "Authorization": "Bearer ${input:aigw-key}"
      }
    }
  }
}

On GitHub, open infra/mcp.json.

Start the server from the mcp.json editor and paste your runtime key when VS Code prompts for it. The inputs entry keeps the key out of the file.

Go keyless

For production apps, ask build to use Microsoft Entra ID instead of a key. Your identity then needs the AI Gateway Models User role to call models, or AI Gateway Tools User to call tools: read access to the gateway grants neither. Signed in to the Azure CLI, you get tokens for the scope https://ai.gateway.azure.com/Data.All. Managed identities and service principals use https://ai.gateway.azure.com/.default. Go keyless with Microsoft Entra ID and RBAC shows both, and how to grant the roles.

Manage the gateway from your agent

If you administer the gateway, the same plugin runs the az aigateway CLI for you. manage takes the gateway name, its resource group, and the task in plain language. Start with a request that only reads:

Text
/aigateway:manage <gateway> <resource-group> list the API keys and the model aliases

It replies with the keys, their state, and their expiration, without their values, and with each alias, its backend model, and the operations it publishes. Ask for changes the same way. manage runs the creates and updates that you ask for, and asks you to confirm before it deletes anything, changes access, or reads a secret. Foundry accounts and Connector Namespaces must already exist. The only thing outside the gateway that manage creates is a role assignment that the gateway needs to work, and only after you confirm. To create a gateway, run /aigateway:create <gateway> <resource-group> <region>.

Clean up

Keep the gateway, model, and alias. Other labs use them.

Delete the key you created for this lab. Then delete the ticket-summarizer folder when you're done. If you added a .env file, it holds a live key.

# Delete the API key that you created for this lab, so that it stops working everywhere
az aigateway api-key delete -g "$RG" --gateway-name "$GATEWAY" -n app-coding-agent --yes &&
  echo "🚮 API key app-coding-agent is deleted"

If a .env file with another gateway key was shared or committed, regenerate that key as shown in Issue and rotate runtime API keys. Remove the ai-gateway-learn entry from .vscode/mcp.json if you added it.

If you did the migration, delete what it created. On the Terraform track, destroy its resource group first:

# Delete the migrated gateway and the resource group that terraform apply created
cd ~/aigateway/labs/17-coding-agent-plugin/terraform && terraform destroy -auto-approve &&
  echo "🚮 Everything that terraform apply created is deleted"

Then delete the sample's resource group, with API Management, the Foundry account, and the gateway of the other tracks, and purge the soft-deleted Foundry account and API Management instance. It takes about 5 minutes.

# Delete the sample's resource group: API Management, the Foundry account, and the migrated gateway
az group delete -n "$MIG_RG" --yes &&
  echo "🚮 Resource group $MIG_RG is deleted"

# Purge the soft-deleted Foundry account, so that its name and quota are free again
az cognitiveservices account purge -g "$MIG_RG" -n "$FOUNDRY" -l "$MIG_LOCATION" &&
  echo "🚮 Foundry account $FOUNDRY is purged"

# Purge the soft-deleted API Management instance
az apim deletedservice purge --service-name "$APIM" --location "$MIG_LOCATION" &&
  echo "🚮 API Management instance $APIM is purged"

If you migrated your own API Management instance, delete only the new gateway, with az aigateway delete --ids "$MIG_GATEWAY_ID" --yes. Then delete the ~/apim-migration folder. Its files hold no secret values, but they describe your instance.

Optionally, remove the plugin. In Cursor, remove it where you installed it.

/plugin uninstall aigateway
/plugin marketplace remove azure

Next steps

Finished the lab?

Mark it complete to track your progress across paths. It's saved only in this browser.