Azure AI Gateway Labs
Documentation Open portal
Agents · Lab 16

Deploy a Foundry hosted agent behind AI Gateway

Deploy your own Microsoft Agent Framework agent to Foundry Hosted Agents with azd. The agent has no model deployment, no direct tool access, and no keys. It calls its model through AI Gateway with its own Microsoft Entra Agent ID, and reaches the Microsoft Learn MCP server through a Foundry toolbox that signs in with the project's managed identity. Run it locally, invoke it in Foundry, and cap its tokens without changing its code.

Intermediate 45 min PortalCLICodeNotebook Updated
Notebook Source Edit

What you'll do

  • Call the gateway without keys, with the agent's Entra Agent ID for models and the project's managed identity for tools
  • Give each identity the AI Gateway User role, in the portal or with the Azure CLI
  • Run the agent locally with azd, then deploy it to Foundry Hosted Agents
  • Cap the agent's tokens with a gateway policy, without changing its code

Before you start

Tracks: Portal, Azure CLI, Code, Notebook

Your team wants to run an agent in Microsoft Foundry, but the platform team needs every model call and every tool call to be governed and visible in one place, with no keys to rotate or leak. In this lab you deploy a small Microsoft Agent Framework agent to Foundry Hosted Agents. Its Foundry project has no model deployments, and the agent holds no secrets: it reaches its model and its tools only through AI Gateway, which authorizes its Microsoft Entra identities with an Azure role.

Before you begin

One-time setupThe Setup guide tasks that this lab uses. Mark each one done once, and the labs stop listing it.

  • Complete Publish a Foundry model and make your first call and Federate MCP tools behind one endpoint. The agent calls the alias from lab 02 and the learn MCP server from lab 04.
  • An Azure subscription where you can create a resource group and a Foundry resource, for example as Owner. azd gives you the Foundry User role on the project it creates, so that you can run the agent locally.
  • Permission to create role assignments on the gateway: Owner, User Access Administrator, or Role Based Access Control Administrator. Contributor can manage the gateway but can't grant roles.
  • The Azure Developer CLI (azd) 1.37.0 or later, and Python 3.13 or later. With uv installed, azd uses it to create the agent's virtual environment.
Warning

azd creates a Foundry resource, and the hosted agent and its model calls are billable. Run Clean up when you're done.

How it works

The agent lives in infra/, a complete azd project:

FileWhat it does
infra/azure.yamlDeclares a Foundry project with no model deployments; gateway-mcp, a connection to the gateway's MCP server; gateway-tools, a Foundry toolbox that exposes that connection; and the hosted agent gateway-agent: Python 3.13, code deploy, 0.5 CPU, 1 GiB, Responses protocol
infra/src/gateway-agent/main.pyThe agent: a model client that points at the gateway, and the toolbox
infra/src/gateway-agent/requirements.txtPinned dependencies, compiled from requirements.in. Foundry builds the agent from this file.

The agent has two outbound paths. Both end at AI Gateway, and neither uses a key:

  • Model. An OpenAIChatClient points at the gateway's model endpoint, /default/models/openai/v1/, and sends your lab 02 alias as model. Instead of a key, it sends a Microsoft Entra token for the gateway's scope, https://ai.gateway.azure.com/.default, as Authorization: Bearer. In Foundry, the token belongs to the agent's own identity. Locally, it belongs to you.
  • Tools. FoundryToolbox calls the gateway-tools toolbox in Foundry. The toolbox reaches the gateway's /default/toolservers/learn/mcp endpoint through the gateway-mcp connection, which signs in with the Foundry project's managed identity for the audience https://ai.gateway.azure.com. The agent never holds a credential for the MCP server.

Three identities call the gateway, and each one needs the AI Gateway User role, which lets it invoke models and call MCP tools:

IdentityShown in AccessEntra ID asCallsExists
The agent's identity, a Microsoft Entra Agent ID<account>-<project>-gateway-agent-AgentIdentity (Service principal)The model endpoint, from FoundryAfter the first deploy
The Foundry project's managed identity<account>/projects/<project> (Service principal)The MCP endpoint, through the connectionAfter provisioning
YouYour account (User)The model endpoint, when the agent runs locallyAlready

So you grant the role in two rounds: the project and yourself before you deploy, then the agent once Foundry has created its identity.

PieceName in this labOwner
Model aliasYour alias from lab 02, in AI_GATEWAY_MODELAI Gateway
MCP serverlearn, at /default/toolservers/learn/mcpAI Gateway
RoleAI Gateway User, caad7a21-134e-43f4-a550-c5ffc17e051c, on the gatewayAI Gateway
Foundry resource and projectIn resource group rg-<env>; the project is named after the azd environmentFoundry, created by azd
Connection and toolboxgateway-mcp and gateway-toolsFoundry, created by azd
Hosted agentgateway-agentFoundry, created by azd

Foundry runs the agent. AI Gateway decides what each identity may call, counts its tokens, and records every call.

Set your variables

Pick one track. You use the same track to grant the roles later.

Choose your track
1

Load the gateway values in your shell

Copy the gateway's runtime URL from its Overview page in the AI Gateway portal, and set the alias from lab 02:

# Set the gateway's runtime URL from its Overview page, and the alias that the agent sends as its model
export AI_GATEWAY_URL="<gateway-url>"
export AI_GATEWAY_MODEL="<model-name>"    # the alias from lab 02
echo "🌐 Runtime URL: $AI_GATEWAY_URL"
echo "🔖 Model alias: $AI_GATEWAY_MODEL"
Checkpoint

Your shell has AI_GATEWAY_URL and AI_GATEWAY_MODEL, and, on the Azure CLI track, GATEWAY_ID.

Configure the azd project

1

Get the lab files

Move into the lab's infra folder and install the Foundry extensions for azd:

Bash
# Move into the lab's azd project
cd ~/aigateway/labs/16-foundry-hosted-agent/infra &&
  echo "📂 Lab folder: ~/aigateway/labs/16-foundry-hosted-agent/infra"

# Install the Foundry extensions for azd, which add the azd ai agent, connection, and toolbox support
azd ext install microsoft.foundry &&
  echo "📦 The Foundry extensions for azd are installed"

microsoft.foundry installs the Foundry extensions for azd, including azure.ai.agents, azure.ai.connections, and azure.ai.toolboxes, the three that azure.yaml requires. If you installed them before, run azd ext upgrade --all.

2

Create an azd environment

The environment name also names the resource group, rg-<env>, and the Foundry project. Choose a region where Foundry Hosted Agents are available, for example eastus2. The environment uses the subscription that the Azure CLI signed in to, the same one as your gateway.

Bash
# Create the azd environment, which also names the resource group, rg-aigw-lab16, and the Foundry project
azd env new aigw-lab16 --location eastus2 --subscription "$(az account show --query id -o tsv)" &&
  echo "📝 azd environment aigw-lab16 deploys to eastus2"
3

Pass the gateway settings to Foundry

azure.yaml uses three azd environment values. The connection gets the MCP endpoint and the gateway's audience, and the agent gets the model endpoint and the toolbox's endpoint, which azd fills in on deploy:

YAML
  gateway-mcp:
    host: azure.ai.connection
    category: RemoteTool
    authType: ProjectManagedIdentity
    audience: https://ai.gateway.azure.com
    target: ${AI_GATEWAY_URL}/default/toolservers/${AI_GATEWAY_MCP_SERVER}/mcp

  gateway-agent:
    env:
      AI_GATEWAY_URL: ${AI_GATEWAY_URL}
      AI_GATEWAY_MODEL: ${AI_GATEWAY_MODEL}
      TOOLBOX_ENDPOINT: ${TOOLBOX_GATEWAY_TOOLS_MCP_ENDPOINT}

Set them from your shell:

# Store the gateway's runtime URL and the alias in the azd environment
azd env set AI_GATEWAY_URL "$AI_GATEWAY_URL" &&
  echo "📝 AI_GATEWAY_URL: $AI_GATEWAY_URL"
azd env set AI_GATEWAY_MODEL "$AI_GATEWAY_MODEL" &&
  echo "📝 AI_GATEWAY_MODEL: $AI_GATEWAY_MODEL"

# Point the connection at the learn MCP server from lab 04
azd env set AI_GATEWAY_MCP_SERVER learn &&
  echo "📝 AI_GATEWAY_MCP_SERVER: learn"

azd keeps the values in .azure/aigw-lab16/.env, which Git ignores. None of them is a secret.

4

Provision the Foundry project

Bash
# Create the resource group, the Foundry resource, and a project with hosted agents enabled
azd provision &&
  echo "🏭 The Foundry project is ready"

It takes about two minutes, and ends with:

Text
SUCCESS: Your application was provisioned in Azure in 1 minute 22 seconds.
🏭 The Foundry project is ready
Checkpoint

azd provision succeeded, and azd env get-value AZURE_AI_PROJECT_ID prints the project's resource ID.

Give the project and yourself access

The project's managed identity calls the MCP endpoint for the toolbox, and you call the model endpoint when you run the agent locally. Grant both AI Gateway User on the gateway, with the track you picked.

Choose your track
1

Find the identities

Print what to look for in the portal: your object ID, and the project's name as the Managed identities tab shows it:

Bash
# Print your object ID, and the name of the Foundry project's managed identity
echo "👤 Your object ID: $(az ad signed-in-user show --query id -o tsv)"
echo "🏭 Foundry project: $(azd env get-value AZURE_AI_ACCOUNT_NAME)/$(azd env get-value AZURE_AI_PROJECT_NAME)"
2

Select both identities

  1. In the AI Gateway portal, open your gateway and select AccessEntra IDAdd role assignment.
  2. On Select identities, in Search by name, email, or object ID, enter your object ID and check your account.
  3. Open the Managed identities tab. In Search managed identities, enter the project's name, and check <account>/<project>, a System-assigned managed identity. The wizard keeps both selections: it shows 2 selected. Select Next.
3

Choose access and scope

  1. On Choose access, select Invoke gateway assets only, then Next.
  2. On Choose scope, select All models and tools, then Next.
  3. On Review, Role assignments to create lists both identities with the role AI Gateway User and the scope All assets. Select Assign.
Checkpoint

AccessEntra ID lists your account and <account>/projects/<project> with the role AI Gateway User, and Models and MCP servers under Access.

Read the agent

main.py is the whole agent. The model client and the toolbox are the only places it talks to the network, and neither holds a key:

Python
credential = DefaultAzureCredential(
    exclude_managed_identity_credential=not os.environ.get("FOUNDRY_HOSTING_ENVIRONMENT"),
)

get_gateway_token = get_bearer_token_provider(credential, AI_GATEWAY_SCOPE)

async def gateway_token() -> str:
    return get_gateway_token()

client = OpenAIChatClient(
    model=model,
    base_url=f"{gateway_url}/default/models/openai/v1/",
    api_key=gateway_token,
)

tools = FoundryToolbox(credential, timeout=60)
  • DefaultAzureCredential signs in as the agent's identity in Foundry, which sets FOUNDRY_HOSTING_ENVIRONMENT in the agent's container. Locally, it skips the managed identity probe and uses your Azure CLI or azd sign-in.
  • The OpenAI SDK calls gateway_token before each request and sends the result as Authorization: Bearer, so the token refreshes on its own.
  • FoundryToolbox reads the toolbox's endpoint from TOOLBOX_ENDPOINT, and signs in to Foundry with the same credential. The toolbox, not the agent, signs in to the gateway's MCP endpoint.
  • HandshakeClient opens the toolbox connection with the MCP initialize handshake. Otherwise the MCP SDK picks the stateless version 2026-07-28, which the toolbox offers, and the toolbox speaks it to the gateway's learn server. That server keeps the versions of its Microsoft Learn backend, so it answers with 403.
  • The instructions ask the model to answer from the search results, and to fetch a whole article only when they don't answer the question. An article is long: with the 10,000 tokens per minute that the quick start and lab 02 give the model deployment, the model call that reads it can fail with 429.

ResponsesHostServer serves the agent on the Responses protocol at port 8088, the same way locally and in Foundry, and creates the agent for each request. The agent sets store to false on its model calls, because the hosting server already keeps the conversation history.

infra/src/gateway-agent/main.pyGitHub
"""A Foundry hosted agent whose model calls and MCP tool calls both go through AI Gateway, without keys.

The agent has no Foundry model deployment and no direct route to a tool:
- Model: it calls the gateway's OpenAI-compatible model endpoint with an alias and a Microsoft Entra token for the
  gateway's audience.
- Tools: it calls a Foundry toolbox, which reaches the gateway's MCP server through a RemoteTool connection that signs
  in with the Foundry project's managed identity.

In Foundry, the tokens come from the agent's managed identity. Locally, `azd ai agent run` uses your Azure CLI sign-in.
To run it without azd, copy .env.example to .env, install requirements.txt in a virtual environment, and run
`python main.py`.
"""

import os

import mcp
from agent_framework import Agent
from agent_framework.openai import OpenAIChatClient
from agent_framework_foundry_hosting import FoundryToolbox, ResponsesHostServer
from azure.identity import DefaultAzureCredential, get_bearer_token_provider
from dotenv import load_dotenv

load_dotenv()


class HandshakeClient(mcp.Client):
    """An MCP client that opens its connection with the initialize handshake.

    By default, the MCP SDK negotiates the stateless MCP version 2026-07-28 when the server offers it, and Foundry
    toolboxes do. The toolbox then speaks 2026-07-28 to the gateway's MCP server, and a server that supports only
    earlier versions, like one whose backend is the Microsoft Learn MCP server, answers with HTTP 403. The handshake
    keeps the whole chain on a version that the gateway's MCP server supports.
    """

    def __init__(self, *args, mode: str = "legacy", **kwargs) -> None:
        super().__init__(*args, mode=mode, **kwargs)


# Agent Framework creates its MCP clients from mcp.Client, so this applies to the toolbox connection.
mcp.Client = HandshakeClient

# The token scope for AI Gateway's runtime endpoints
AI_GATEWAY_SCOPE = "https://ai.gateway.azure.com/.default"

INSTRUCTIONS = """
You answer questions about Microsoft and Azure technologies.
- Search the Microsoft Learn tools before you answer, and cite the URLs you used.
- Answer from the search results. Fetch a whole article only when they don't answer the question: articles are long,
  and their tokens count toward the model's limits.
- If the tools don't cover the question, say so instead of guessing.
- Keep answers under 150 words.
"""


def required(name: str) -> str:
    value = os.environ.get(name, "").strip()
    if not value:
        raise RuntimeError(f"Set {name}. With azd, run: azd env set {name} <value>")
    return value


# Foundry sets FOUNDRY_HOSTING_ENVIRONMENT in the agent's container. Locally, skip the managed identity probe, so that
# the credential goes straight to your Azure CLI or azd sign-in.
credential = DefaultAzureCredential(
    exclude_managed_identity_credential=not os.environ.get("FOUNDRY_HOSTING_ENVIRONMENT"),
)


def create_agent() -> Agent:
    gateway_url = required("AI_GATEWAY_URL").rstrip("/")
    model = required("AI_GATEWAY_MODEL")
    get_gateway_token = get_bearer_token_provider(credential, AI_GATEWAY_SCOPE)

    async def gateway_token() -> str:
        return get_gateway_token()

    # The SDK sends the token as Authorization: Bearer, and asks for a fresh one before each call.
    client = OpenAIChatClient(
        model=model,
        base_url=f"{gateway_url}/default/models/openai/v1/",
        api_key=gateway_token,
    )

    # FoundryToolbox reads the toolbox endpoint from TOOLBOX_ENDPOINT, and signs in to Foundry with the same credential.
    tools = FoundryToolbox(credential, timeout=60)

    return Agent(
        client=client,
        name="gateway-docs-agent",
        instructions=INSTRUCTIONS,
        tools=tools,
        # Foundry keeps the conversation history, so the model call doesn't need to store it.
        default_options={"store": False},
    )


if __name__ == "__main__":
    # The server creates the agent for each request, and closes its toolbox connection afterward.
    ResponsesHostServer(agent=create_agent).run()

On GitHub, open main.py and azure.yaml.

Deploy to Foundry

1

Deploy the connection, the toolbox, and the agent

Bash
# Create the MCP connection and the toolbox, upload the agent's code, and create the hosted agent
azd deploy --all &&
  echo "🚀 The hosted agent gateway-agent is deployed to Foundry"

azd creates the gateway-mcp connection, gives you Foundry User on the project for the toolbox, creates the gateway-tools toolbox, uploads src/gateway-agent as a code package, and creates the hosted agent. Foundry builds it from requirements.txt and gives it its own identity. It takes about three minutes:

Text
  gateway-mcp: Deploying (Upserting connection "gateway-mcp") [1s]
  gateway-tools: Deploying (Assigning Foundry User on the toolbox's Foundry project.) [9s]
  gateway-tools: Deploying (Upserting toolbox "gateway-tools") [31s]
  gateway-agent: Deploying (Creating new agent from code package) [34s]
  gateway-agent: Done [2m38s]
...
SUCCESS: Your application was deployed to Azure in 2 minutes 38 seconds.
🚀 The hosted agent gateway-agent is deployed to Foundry

The first deploy can also print WARNING: gateway-agent (predeploy): AZURE_CONTAINER_REGISTRY_ENDPOINT not set; skipping ACR role check. The agent is deployed as code, so the deploy still succeeds.

Give the agent access

Foundry created the agent's identity on deploy. It stays the same when you deploy new versions, so you grant it once. Use the track you picked.

Choose your track
1

Find the agent's identity

Bash
# Print the object ID of the agent's identity, which azd saved on deploy
echo "🤖 Agent identity: $(azd env get-value AGENT_GATEWAY_AGENT_INSTANCE_IDENTITY_PRINCIPAL_ID)"
2

Assign the role

  1. Select AccessEntra IDAdd role assignment.
  2. On Select identities, in Search by name, email, or object ID, enter the agent's object ID. Check <account>-<project>-gateway-agent-AgentIdentity, a Service principal, and select Next.
  3. On Choose access, select Invoke gateway assets only, then Next.
  4. On Choose scope, select All models and tools, then Next.
  5. On Review, select Assign.

The role takes a couple of minutes to apply. Run the agent locally in the meantime.

Checkpoint

AccessEntra ID lists three identities with AI Gateway User: you, the project, and <account>-<project>-gateway-agent-AgentIdentity.

Run the agent locally

1

Start the agent

From the infra folder:

Bash
# Start the agent locally: azd installs requirements.txt in a virtual environment and opens Agent Inspector
echo "🚀 Starting the agent on http://localhost:8088 (Ctrl+C to stop)"
azd ai agent run

azd creates a virtual environment, installs requirements.txt, starts the agent on http://localhost:8088 with the values from the azd environment, and opens Agent Inspector, where you can chat with the agent and see its tool calls. Add --no-client to skip Agent Inspector.

2

Ask a question

In a second terminal, from the same folder:

Bash
# Ask the local agent a question: it calls the model as you, and the Learn tools through the toolbox
echo "🤖 The local agent's answer:"
azd ai agent invoke --local "Find the Microsoft Learn article that explains Azure API Management key concepts. Reply with its title and URL in one sentence."

Expected output: an answer that cites a learn.microsoft.com URL, for example:

Text
🤖 The local agent's answer:
Target: localhost:8088 (local)
...
[local] The Microsoft Learn page "What is Azure API Management?" (Key concepts) explains Azure API Management key concepts: https://learn.microsoft.com/azure/api-management/api-management-key-concepts

Behind that answer, the agent made a model call that asked for a tool, one or more Learn tool calls through the toolbox, and a model call that wrote the answer, all through the gateway. The model calls carried your token, and the tool calls carried the project's. Stop the agent with Ctrl+C.

Checkpoint

The local agent answers with a Microsoft Learn URL.

Invoke the hosted agent

1

Ask the hosted agent

Bash
# Ask the hosted agent the same question in Foundry, where it calls the model with its own identity
echo "🤖 The hosted agent's answer:"
azd ai agent invoke gateway-agent "Find the Microsoft Learn article that explains Azure API Management key concepts. Reply with its title and URL in one sentence."

Expected output: the same kind of answer, now from Foundry. The first call can take 30 seconds or more while the agent starts.

Text
🤖 The hosted agent's answer:
Agent:        gateway-agent (remote)
...
[gateway-agent] The Microsoft Learn article is "What is Azure API Management?" — https://learn.microsoft.com/azure/api-management/api-management-key-concepts

azd keeps the conversation, so a second invoke, such as "Shorten that to ten words.", continues it. Add --new-conversation to start over.

2

Read the agent's logs

When a hosted call fails, azd prints only ERROR: agent failed (server_error): An internal server error occurred. The agent's logs show the gateway's answer:

# Fetch the last 300 log lines of the agent's latest session, and keep the errors from the gateway
echo "📊 Errors in the agent's logs:"
azd ai agent monitor gateway-agent --tail 300 | grep "Error code"

If the agent's role hasn't applied yet, the logs show a 403:

Text
... openai.PermissionDeniedError: Error code: 403 - Access denied. You do not have permission to access the requested resource. Ask an administrator to grant the required Azure role.

Wait a minute and invoke again. A successful call prints nothing here.

3

Find the agent in Foundry

Bash
# Show the hosted agent's status, its identity, its Responses endpoint, and its Playground URL
echo "📊 Hosted agent gateway-agent:"
azd ai agent show gateway-agent

The output shows Status active, the Instance Identity Principal ID that you granted the role, the agent's Responses endpoint, and a Playground URL. Open that URL to chat with the agent in the Foundry portal.

Checkpoint

azd ai agent show gateway-agent reports active, and azd ai agent invoke returns an answer with a Learn URL.

See the agent in Monitoring

Each identity is a caller identity of its own. With monitoring set up as in Monitor usage, cost, and traces, invoke the agent a few times, wait a few minutes, then open Monitoring in the portal:

  • Models: Model usage by caller identity counts the agent's identity apart from you and your other apps, with its calls, tokens, and estimated cost for your alias.
  • Tools: the learn tool server under Usage by tool server, with the tools the toolbox called under Top tools within tool servers. The toolbox makes those calls, so they belong to the project's identity.
  • Traces: one row per model or tool call, with the policies the gateway applied.

Cap the agent's tokens

Give the agent's alias a strict token limit counted per caller identity, then watch the gateway enforce it without any change to the agent. Use the track you picked.

Choose your track
1

Add the token limit

  1. Select PoliciesAdd policy.
  2. On Type, select Token rate limit.
  3. On Scope, select Individual assets and pick your alias.
  4. On Details, keep Time period at Per minute, set Allowed tokens to 1000, keep Counter dimensions at Identity, and select Create.
1

Trigger the limit

Wait about a minute for the policy to apply, then invoke the hosted agent twice:

Bash
# Invoke the hosted agent twice, each time in a new conversation: the first invoke uses well over 1,000 tokens
echo "🚦 Expect the second invoke to fail with a 429 from the gateway's token limit"
azd ai agent invoke gateway-agent --new-conversation "Summarize Azure API Management in three sentences with Learn links."
azd ai agent invoke gateway-agent --new-conversation "Summarize Azure API Management in three sentences with Learn links."

The gateway counts tokens when a call completes, so the first invoke usually succeeds and uses well over 1,000 tokens, because Learn tool results are long. The second invoke fails with ERROR: agent failed (server_error): An internal server error occurred., and the agent's logs show the gateway's 429:

# Fetch the agent's latest logs, and keep the errors from the gateway
echo "📊 Errors in the agent's logs:"
azd ai agent monitor gateway-agent --tail 300 | grep "Error code"
Text
📊 Errors in the agent's logs:
... openai.RateLimitError: Error code: 429 - Token limit is exceeded. Try again in 431 seconds.
...

The counter belongs to the agent's identity. Run the agent locally and it still answers: locally, it signs in as you, a different caller identity with its own counter. The wait in the message can be several minutes, because the first invoke spent several times the 1,000-token budget. For more on limits, see Throttle tokens per caller with token limits.

2

Remove the limit

In the portal, select Policies. On the row of the Token rate limit for your alias, select More actionsDelete, and confirm with Delete. With the CLI, remove the tokenLimit policy the same way you added it:

# Write the alias's policies, without the token limit, to a file
az aigateway model-alias show -g "$RG" --gateway-name "$GATEWAY" -n "$ALIAS_NAME" -o json \
  | jq -c '[(.properties.policies // [])[] | select(.type != "tokenLimit") | del(.scope)]' > policies.json

# Replace the alias's whole policy array with the file's, then delete the file
az aigateway model-alias update -g "$RG" --gateway-name "$GATEWAY" -n "$ALIAS_NAME" --policies @policies.json \
  --query properties.policies &&
  echo "🚮 The token limit is removed from $AI_GATEWAY_MODEL"
rm policies.json

If your alias had its own token limit before this lab, the CLI commands remove it too: add it back with the count it had. When the alias has no policies left, the update prints no policies, only the 🚮 line.

Verify

Verify
  • azd ai agent invoke --local and azd ai agent invoke both return an answer with a Microsoft Learn URL.
  • The Foundry project has no model deployments and the azd environment holds no gateway key, yet the agent answers: its model and tool calls go through the gateway with Microsoft Entra tokens.
  • AccessEntra ID lists the agent's identity, the project, and you with AI Gateway User.
  • With the token limit in place, the agent got 429 from the gateway without any code change.

Go further

  • Narrow the roles. AI Gateway User covers models and tools. The agent's identity only calls the model, and the project's managed identity only calls the MCP server, so assign them AI Gateway Models User and AI Gateway Tools User instead: in the wizard, pick All models or All tools as the scope. Running locally, you need both. Go keyless with Microsoft Entra ID and RBAC scopes roles further, to one alias or MCP server.
  • Use an API key where Entra isn't an option. Create a key in AccessAPI keys, and send it as the SDK's API key, which the SDK sends as Authorization: Bearer <key>. MCP clients send the same header. Don't add an api-key header as well: the gateway rejects a request that has both with 400 and "Specify either an API key or an Authorization header, not both".
  • Lock down egress. Foundry can restrict where a hosted agent may connect. Allow only your gateway's host name, so the gateway is the agent's only way out. See the egress control sample.
  • Use another framework. Swap the agent's internals for the OpenAI Agents SDK or LangGraph, as in Connect agent frameworks to models and tools. The gateway settings stay the same.
  • Skip the agent code. A gateway connection gives a Foundry project's playground and prompt agents the gateway's models, with no code to deploy, as in Use gateway models in Microsoft Foundry projects.
  • Charge the agent back. Attribute its tokens and estimated cost with Attribute tokens and cost with OpenTelemetry.

Clean up

Remove the roles of the agent and the project first, while their identities still exist, then delete the Foundry resources. Use the track you picked.

Choose your track
1

Remove the role assignments

In AccessEntra ID, on the rows of <account>-<project>-gateway-agent-AgentIdentity and <account>/projects/<project>, select More actions, then Remove role assignment, and confirm with Remove.

Keep your own AI Gateway User role if you continue with other labs. Go keyless with Microsoft Entra ID and RBAC shows how to remove it.

Delete the Foundry resources

Delete the Foundry resource group, with the connection, the toolbox, and the hosted agent. --purge also purges the soft-deleted Foundry resource so you can reuse its name:

Bash
# Delete the resource group with the Foundry resource and the hosted agent, and purge the soft-deleted resource
azd down --purge --force &&
  echo "🚮 The Foundry resource group and the hosted agent are deleted"

Delete the .azure folder in infra if you no longer need the environment. Your gateway, its alias, and the learn MCP server aren't affected. To do the lab again soon after, create the azd environment with a new name, such as aigw-lab16b: in a Foundry project that you recreate with the same name, azd deploy --all can fail with get_toolbox: the Foundry project has not synchronized.

Next steps

Finished the lab?

Mark it complete to track your progress across paths. It's saved only in this browser.