Azure AI Gateway Labs
Documentation Open portal
Models & routing · Lab 03

Publish models from any provider behind stable aliases

Connect Anthropic or another non-Azure provider with a credential that stays in the gateway, publish its models behind callable aliases, call Claude with the native Messages API, and retarget an alias to a different backend model without changing a single client.

Intermediate 30 min PortalCLIBicepTerraform Updated
Source Edit

What you'll do

  • Add a custom model provider whose API key stays in the gateway
  • Publish backend models behind gateway-wide callable aliases
  • Call an Anthropic model through the gateway with the native Messages API
  • Give one backend model several aliases, each with its own token limit
  • Retarget an alias to another backend model without changing clients or losing its policies
  • Declare the provider, its models, and the aliases with Bicep and azd, with the provider key as a secure parameter

Before you start

Tracks: Portal, Azure CLI, Bicep + azd, Terraform

The support team wants to try Claude, and the batch team wants the same model for nightly summaries. You don't want either team to hold an Anthropic key, and you don't want to edit their code every time you switch models. In this lab you add Anthropic as a provider whose key only the gateway knows, publish two Claude models behind aliases such as support-chat, and then point support-chat at a bigger model while the clients keep sending the same name.

Before you begin

One-time setupThe Setup guide tasks that this lab uses. Mark each one done once, and the labs stop listing it.

Tip

On the Bicep + azd and Terraform tracks, azd up or terraform apply creates its own gateway, Foundry model, and key, so you can skip this section, except for the Anthropic key. You need only that key and the one-time setup above.

  • Complete Publish a Foundry model and make your first call. You need the gateway, a runtime API key, and the Foundry alias you created there.
  • An API key from the Claude Console that can call two models, for example claude-haiku-5-5 and claude-sonnet-5-5. To use another provider instead, see the values in Add Anthropic as a custom provider.
  • Permission to change the gateway, for example Contributor on its resource group.
  • Run the scripts and samples from the lab's folder, ~/aigateway/labs/03-multi-provider-aliases.

Set these variables once. Every track and the Verify section reuse them.

# Name your gateway, the provider that you create in this lab, and the two Anthropic models
export RG="<resource-group>"
export GATEWAY="<gateway>"
export PROVIDER="anthropic"               # the gateway provider you create in this lab
export FAST_MODEL="claude-haiku-5-5"      # two models that your Anthropic key can call
export SMART_MODEL="claude-sonnet-5-5"
echo "📝 Gateway $GATEWAY in $RG, provider $PROVIDER, models $FAST_MODEL and $SMART_MODEL"

# Read the gateway's runtime URL, which apps call
export AI_GATEWAY_URL="https://$(az aigateway show -g "$RG" -n "$GATEWAY" --query properties.frontend.defaultHostname -o tsv)"
echo "🌐 Runtime URL: $AI_GATEWAY_URL"

# Paste the gateway runtime key from lab 02, not the Anthropic key. Don't commit it or write it to a file.
printf "AI Gateway API key: " && read -rs AI_GATEWAY_API_KEY && export AI_GATEWAY_API_KEY && echo
echo "🔑 Key: in AI_GATEWAY_API_KEY (${#AI_GATEWAY_API_KEY} characters, not shown)"

How it works

The gateway publishes models in three layers:

ObjectWhat it holdsUnique withinExample
Model providerA backend endpoint and the credential the gateway uses to call itThe gatewayanthropic, key sent in x-api-key
Backend modelA model that the provider serves, and the interfaces it supportsIts providerclaude-haiku-5-5
AliasThe callable name that clients put in the model field. It points to one or more backend models and can carry its own policies.The gatewaysupport-chat

Clients only ever see the gateway URL, their runtime key, and the alias. The provider key stays in the gateway. In this lab each alias points to one backend model. To split traffic across several backend models by priority and weight, see Route an alias across models by priority and weight.

Each interface has its own route under <gateway-url>/default/models:

InterfaceRoute
OpenAI Chat Completions/openai/v1/chat/completions
OpenAI Responses/openai/v1/responses
Anthropic Messages/anthropic/v1/messages

Every model runs in passthrough mode. The gateway keeps the provider's request and response format and doesn't translate between OpenAI and Anthropic, so call each model on an interface it advertises, and retarget an alias only to a model that supports the same interface.

Add Anthropic as a custom provider

The gateway connects to these providers. Microsoft Foundry and Azure OpenAI can use the gateway's managed identity. AWS Bedrock has its own wizard. Every other provider is a custom provider. For both, the gateway stores the API key.

ProviderHow you add itGateway authenticates with
Microsoft FoundryModels from Microsoft FoundryManaged identity (recommended) or API key
Azure OpenAIModels from Microsoft FoundryManaged identity (recommended) or API key
OpenAIModels from a custom providerAPI key
AnthropicModels from a custom providerAPI key
AWS BedrockModels from AWS BedrockBedrock API key (the gateway adds the Bearer prefix)
Google Vertex AIModels from a custom providerAPI key
Any other OpenAI-compatible endpointModels from a custom providerAPI key

Pick one track. Every track creates the same result: a provider named anthropic, two backend models, and two aliases. support-chat points to the fast model, and anthropic/claude-sonnet-5-5 points to the bigger one. The Bicep + azd and Terraform tracks also declare the batch-chat alias and the token limits of the next section.

Choose your track
1

Start the custom provider wizard

Open the AI Gateway portal and select your gateway from the gateway switcher. Select ModelsAddModels from a custom provider. Add custom provider in the Set up your gateway guide on Home opens the same wizard.

On the Provider step, select Create new provider and fill in Basic info:

FieldValue
Display nameAnthropic
Provider nameanthropic, which the wizard fills in from the display name. It's used in the resource ID, and you can't change it later.
DescriptionOptional

Select Next.

2

Connect the endpoint

On the Endpoint step, fill in Endpoint and authentication:

FieldValue for Anthropic
Base endpoint URLhttps://api.anthropic.com
Header namex-api-key, in place of the default Authorization
API keyYour Anthropic API key
Authentication schemeLeave Include Bearer scheme in the authorization header value cleared. Anthropic expects the raw key.

The wizard lists the provider's models by calling <base endpoint URL>/v1/models, so leave /v1 off the base URL. The key is stored in the gateway. Apps that call the gateway never receive it.

For other providers, change these values:

ProviderBase endpoint URLHeader nameAuthentication scheme
OpenAIhttps://api.openai.comAuthorizationSelected, so the gateway sends Bearer <key>
Other OpenAI-compatible endpointsThe URL in front of /v1/...The header the endpoint expectsAs the endpoint expects
Google Vertex AIThe provider's OpenAI-compatible endpoint, without the /v1/... pathThe header that the provider's API key goes inAs the provider expects

Select Next.

3

Choose the models and their aliases

The Models step calls the provider to discover its models. Select AddIndividual model, fill in the Add model dialog for each model, and select Add:

FieldFirst modelSecond model
Backend modelclaude-haiku-5-5claude-sonnet-5-5
Create an aliasSelectedSelected
AliasChange the default, anthropic/claude-haiku-5-5, to support-chatKeep the default, anthropic/claude-sonnet-5-5
API formatAnthropic Messages. Provider path fills in as /v1/messages.Anthropic Messages

If the step shows No models discovered., check the base URL and the key. You can still add the models by hand. When Included lists both models, select Add models.

Checkpoint

Models lists the two Claude models of the Anthropic provider and the aliases support-chat and anthropic/claude-sonnet-5-5. Open a model: under Endpoints, its Overview shows the Messages endpoint, <gateway-url>/default/models/anthropic/v1/messages.

Call Claude through the gateway

Send a native Anthropic Messages request to the gateway's Anthropic route. Three things differ from a direct call to Anthropic: the URL is the gateway's, Authorization: Bearer carries your gateway runtime key, and model is the alias. Keep the anthropic-version header that the Messages API expects.

# Call support-chat with the native Messages API: your runtime key as a bearer token, and the alias in model
curl -sS "${AI_GATEWAY_URL%/}/default/models/anthropic/v1/messages" \
  -H "Authorization: Bearer $AI_GATEWAY_API_KEY" \
  -H "anthropic-version: 2023-06-01" \
  -H "Content-Type: application/json" \
  -d '{"model": "support-chat", "max_tokens": 1024,
       "messages": [{"role": "user", "content": "In one sentence, what is an AI gateway?"}]}' \
  | jq -r 'if .content then
             "✨ Backend model: \(.model)",
             "🤖 \([.content[] | select(.type == "text") | .text] | join(""))",
             "📊 Usage: \(.usage | tojson)"
           else "🚨 \(.error.message // .)" end'

The response uses Anthropic's own format, with content blocks, stop_reason, and usage.input_tokens. It doesn't use OpenAI's choices. The model field names the backend model that answered, for example claude-haiku-5-5. Claude Sonnet 5.5 and Haiku 5.5 think adaptively by default, so content can start with a thinking block, and thinking counts toward max_tokens. Read the text blocks and leave headroom in max_tokens.

Checkpoint

The call prints a 🤖 Claude reply and ✨ Backend model: claude-haiku-5-5. Your app never sent an Anthropic key.

Tip

To stream the reply, add "stream": true to the body and pass -N to curl. The gateway answers with text/event-stream and forwards Anthropic's events as they arrive: message_start, content_block_delta, message_delta, and message_stop. Streaming Anthropic Messages through the gateway works since the November 6 release.

Give one backend model two aliases

The batch team's nightly job shouldn't be able to use up the capacity that the support app needs. Point a second alias, batch-chat, at the same fast model and give each alias its own per-caller token limit: 2,000 tokens per minute for batch-chat and 20,000 for support-chat. Alias policies apply only to callers that use that name. Policies on the backend model apply to every alias that targets it.

Choose your track
1

Add the batch-chat alias

Select ModelsAliasesAdd alias. The Create alias wizard has seven steps:

StepWhat to do
BasicsSet Alias name to batch-chat and Description to Nightly batch summaries.
Auto model selection (preview)Leave Choose a model based on the prompt off.
Select modelsSelect claude-haiku-5-5 from the Anthropic provider. The models of one alias must share an API format.
Session affinityKeep the default. Session affinity needs several backend models.
Priorities and weightsKeep the defaults, priority 0 and weight 1. With one backend model, it gets all requests.
Failure handlingKeep the default, with no failure rules.
ReviewCheck the summary.

Select Create. You can also start from ModelsAddAlias for existing models.

2

Limit each alias

Select PoliciesAdd policy. On the Type step, select Token rate limit under Cost and rate limits, then Next. On the Scope step, select Individual assets, select the batch-chat alias, then Next. On the Details step, set Time period to Per minute, Allowed tokens to 2000, and Counter dimensions to Identity. Select Create.

Repeat for the support-chat alias with 20000 Allowed tokens.

Checkpoint

In ModelsAliases, open batch-chat and select its Policies tab. It lists a Token rate limit with the configuration 2,000 tokens / minute per caller identity and the scope This alias. The support-chat alias lists 20,000 tokens / minute per caller identity.

Retarget an alias without touching clients

The support team wants better answers. Point support-chat at the bigger model. The clients keep sending "model": "support-chat", and the alias keeps its 20,000-token limit.

Choose your track
1

Edit the alias

In ModelsAliases, open the support-chat alias and select Edit. Edit alias shows the sections of Create alias on one page. Under Select models, clear claude-haiku-5-5 and select claude-sonnet-5-5 from the Anthropic provider, then select Save. New requests use the selected backend model within about 30 seconds.

Edit alias changes the backend models behind the name. The alias name that clients send stays the same. With several backend models, Priorities and weights decides which one serves each request, as in lab 22.
2

Check that the policy stayed

On the support-chat alias page, select the Policies tab.

Checkpoint

The alias page shows claude-sonnet-5-5 on the Backend models tab, and the Policies tab still lists the 20,000-token Token rate limit.

Note

Retarget only between backend models that support the same interface. An alias that clients call with the Anthropic Messages API must keep pointing to a model that supports Anthropic Messages.

Verify

1

List the aliases

# List the aliases, with their backend models and token limits
echo "🔖 Aliases on $GATEWAY:"
./infra/alias.sh list

Expected output (the order and your Foundry alias differ):

Text
🔖 Aliases on <gateway>:
ALIAS                        PROVIDER/BACKEND MODEL       ALIAS POLICIES
anthropic/claude-sonnet-5-5  anthropic/claude-sonnet-5-5  -
batch-chat                   anthropic/claude-haiku-5-5   tokenLimit 2000/minute
contoso-ai/gpt-5-mini        contoso-ai/gpt-5-mini        -
support-chat                 anthropic/claude-sonnet-5-5  tokenLimit 20000/minute

Your Foundry and Anthropic models sit side by side behind one gateway. In the portal, Models shows the same list.

2

Call both aliases

The script sends a Messages request and prints the status, the backend model that answered, and the reply.

# Call each alias with the same Messages request, and print the backend model that answered
echo "🤖 Replies through support-chat and batch-chat:"
./infra/call-anthropic.sh support-chat
./infra/call-anthropic.sh batch-chat

On GitHub, open call-anthropic.sh or call-anthropic.ps1.

Expected output (the replies differ):

Text
🤖 Replies through support-chat and batch-chat:
Alias:   support-chat
Status:  200
Model:   claude-sonnet-5-5
Reply:   An AI gateway is a control point between applications and AI models that ...
Alias:   batch-chat
Status:  200
Model:   claude-haiku-5-5
Reply:   An AI gateway sits between your apps and AI models to ...
Checkpoint

support-chat now answers with Sonnet and batch-chat with Haiku, and you didn't change the client request.

3

Confirm the alias kept its policy

# Show the policies on support-chat, which kept its token limit when you retargeted it
echo "🔐 Policies on support-chat:"
./infra/alias.sh show support-chat | jq '.properties.policies'

The output lists the tokenLimit policy with "count": 20000 and "counterKey": ["identity"]. In the portal, open support-chat and select Policies.

Go further

  • Use the Anthropic SDK. Point the SDK at the gateway's Anthropic base URL, and pass the runtime key as auth_token, which the SDK sends as Authorization: Bearer. The SDK appends /v1/messages itself and sends anthropic-version for you.

    Python
    import os
    from anthropic import Anthropic  # pip install anthropic
    
    client = Anthropic(
        base_url=os.environ["AI_GATEWAY_URL"].rstrip("/") + "/default/models/anthropic",
        auth_token=os.environ["AI_GATEWAY_API_KEY"],  # sent as Authorization: Bearer
    )
    message = client.messages.create(
        model="support-chat",
        max_tokens=1024,
        messages=[{"role": "user", "content": "Hello"}],
    )
    print(message.model, "".join(block.text for block in message.content if block.type == "text"))
  • Move one alias between providers. Add OpenAI as a second custom provider with the values from the provider table, and register gpt-5-mini with the OpenAI Chat Completions API format. Create an alias such as team-chat on your Foundry gpt-5-mini, then retarget it to the OpenAI model. Both speak Chat Completions, so clients don't notice the switch.

  • Layer the limits. Add a token limit to the claude-haiku-5-5 backend model as in Throttle tokens per caller with token limits. It applies to every alias that targets the model, on top of each alias's own limit.

  • Read the product docs. Model API formats and providers lists the supported APIs and providers, and Create and update a model alias covers every step of the alias wizard.

Clean up

Keep the gateway, the runtime key, and the Foundry model from lab 02. Later labs use them.

Choose your track
1

Delete the aliases, models, and provider

Remove what you added in this lab in this order: aliases, then backend models, then the provider. Open each of the support-chat, batch-chat, and anthropic/claude-sonnet-5-5 aliases in Models and delete them. Then delete the two Claude backend models and the Anthropic provider.

Deleting the provider removes the stored Anthropic key from the gateway. It doesn't revoke the key itself. If you created the key only for this lab, delete it in the Claude Console.

Next steps

Finished the lab?

Mark it complete to track your progress across paths. It's saved only in this browser.