Azure AI Gateway Labs
Documentation Open portal
Agents · Lab 24New

Use your gateway's models and tools in GitHub Copilot

Point GitHub Copilot in VS Code, the Copilot CLI, the Copilot SDK, and the GitHub Copilot app at gpt-6.1-sol published by your gateway, then give them Microsoft Learn tools from an MCP server on the same gateway, so every model and tool call uses your key, your limits, and your monitoring.

Intermediate 40 min PortalCLIBicepTerraformCodeNotebook Updated
Notebook Source Edit

What you'll do

  • Publish gpt-6.1-sol through the gateway with a per-caller token limit and a runtime key for GitHub Copilot
  • Add the gateway as a custom endpoint in VS Code and run agent mode on gpt-6.1-sol
  • Run the Copilot CLI against the gateway with provider environment variables
  • Start a Copilot SDK session in Python or TypeScript that sends every model call to the gateway
  • Add the gateway as a model provider in the GitHub Copilot app
  • Publish the Microsoft Learn MCP server through the gateway for Copilot's agents
  • Give VS Code, the Copilot CLI, the Copilot SDK, and the GitHub Copilot app the gateway's Learn tools with the same key
  • Track Copilot usage by key in Monitoring and cut off access by suspending the key

Before you start

Tracks: Portal, Azure CLI, Bicep + azd, Terraform, Code, Notebook

GitHub Copilot can run on a model that you bring yourself: give it an OpenAI-compatible endpoint and a key, and it sends every chat and agent request there. Make that endpoint your gateway, and Copilot's traffic gets the same controls as your apps: a key that you can suspend, a token limit for each caller, and usage in Monitoring. Copilot never sees a Foundry credential, because the gateway calls Foundry with its managed identity. In this lab you publish gpt-6.1-sol for Copilot, then connect four Copilot surfaces to it: VS Code, the Copilot CLI, the Copilot SDK, and the GitHub Copilot app. Then you publish the Microsoft Learn MCP server through the same gateway and give those surfaces its tools, with the same key, so Copilot's tool calls get the same controls as its model calls.

Before you begin

One-time setupThe Setup guide tasks that this lab uses. Mark each one done once, and the labs stop listing it.

Tip

On the Bicep + azd and Terraform tracks, azd up or terraform apply creates its own gateway, Foundry account, gpt-6.1-sol alias, key, and learn MCP server, and the track sets the variables, so you can skip lab 02 and the variables below. You need the one-time setup above, plus quota for the model, a Copilot surface, and the practice project.

  • Complete Publish a Foundry model and make your first call. You need a gateway with a Foundry provider, and permission to change it, for example Contributor on the resource group.

  • Quota for gpt-6.1-sol (version 2026-09-29) in the Foundry account behind your gateway. This lab deploys it with 100,000 tokens per minute (TPM), because coding agents send large prompts. Model inference is billed per use.

  • At least one Copilot surface. You don't need all four:

    SurfaceVersion testedNeeds
    VS Code with GitHub Copilot1.143Nothing else. Chat and agent mode work with your own model without a GitHub sign-in.
    GitHub Copilot CLI1.0.112Nothing else. A GitHub sign-in is optional with your own model.
    GitHub Copilot SDK1.0.34 (TypeScript), 1.0.31 (Python)Node.js 22.18 or later, or Python 3.11 or later, and a clone of this repository.
    GitHub Copilot appPublic previewA GitHub sign-in. A Copilot plan isn't required for your own models.
  • Run the scripts and samples from the lab's folder, ~/aigateway/labs/24-github-copilot-byok.

Important

On Copilot Business or Copilot Enterprise, an administrator must allow custom models before VS Code and the Copilot app offer them. See Enable custom models for your enterprise.

Set these variables once. Every track and the Verify section reuse them.

# Name your gateway, the Foundry account behind it, and what this lab publishes on the gateway
export RG="<resource-group>"
export GATEWAY="<gateway>"
export FOUNDRY_RG="<foundry-resource-group>"      # resource group of the Foundry account behind the gateway
export FOUNDRY_ACCOUNT="<foundry-account-name>"   # the account name, not a project or deployment name
export PROVIDER="foundry-models"                  # the gateway's Foundry provider from lab 02
export MODEL="gpt-6.1-sol"                        # the deployment, the backend model, and the alias
export KEY_NAME="github-copilot"                  # the runtime key that Copilot uses

# Read the gateway's resource ID and runtime URL
export GATEWAY_ID="$(az aigateway show -g "$RG" -n "$GATEWAY" --query id -o tsv)"
export AI_GATEWAY_URL="https://$(az aigateway show -g "$RG" -n "$GATEWAY" --query properties.frontend.defaultHostname -o tsv)"
export AI_GATEWAY_MODEL="$MODEL"
echo "📝 Gateway $GATEWAY in $RG, with Foundry account $FOUNDRY_ACCOUNT in $FOUNDRY_RG"
echo "🌐 Runtime URL: $AI_GATEWAY_URL"
echo "🔖 Model alias: $AI_GATEWAY_MODEL"
Tip

If you imported models in the portal in lab 02, your provider has another name. List the providers with az aigateway model-provider list -g "$RG" --gateway-name "$GATEWAY" --query "[].name" -o tsv and set PROVIDER to the one for your Foundry account.

Then create a small practice project with two deliberate problems: add subtracts, and divide doesn't guard against zero. The VS Code and CLI sections ask Copilot to work on it. Create it outside any repository:

# Create the practice project's folder, and write calc.py with its two deliberate problems
mkdir -p ~/contoso-calc
cat > ~/contoso-calc/calc.py <<'EOF'
"""Tiny calculator used to try GitHub Copilot with a model from Azure AI Gateway."""


def add(a: float, b: float) -> float:
    return a - b


def divide(a: float, b: float) -> float:
    return a / b


if __name__ == "__main__":
    print(add(2, 3), divide(10, 2))
EOF
echo "📄 Practice project is ready: ~/contoso-calc/calc.py"

How it works

Bring your own key (BYOK) replaces the model endpoint behind Copilot. Copilot still runs its agent loop on your machine: it reads files, plans, calls tools, and asks before it edits or runs commands. Each model request goes to the gateway instead of GitHub's model service. Every surface needs the same four settings:

SettingValueWhy
Base URLhttps://<gateway-host>/default/models/openai/v1The gateway's OpenAI-compatible endpoint. VS Code wants the full URL, with /responses at the end.
Modelgpt-6.1-solThe alias. Copilot sends it in the request's model field, and the gateway routes by alias. When the alias has the model's well-known name, Copilot also applies the prompts and token sizes it uses for that model.
APIResponsesCopilot recommends the Responses API for GPT-5 and later models. The gateway serves both /responses and /chat/completions for this model.
API keyThe github-copilot runtime keyCopilot sends it as Authorization: Bearer, the header the gateway reads keys from.

Two things differ from an app that calls the gateway:

  • You add the model by hand. Clients often call GET /models to discover what an endpoint offers, but the gateway doesn't serve a model list. Each surface lets you type the model name instead.
  • Agents send big requests. Every request carries Copilot's system prompt, its tool definitions, and the conversation so far. In testing, a one-line question from the Copilot CLI used 14,000 to 38,000 input tokens. That's why this lab gives each caller 100,000 tokens per minute, and why one key per tool or per developer makes usage easy to read.

Tools follow the same pattern. Later in this lab, the gateway also publishes the Microsoft Learn MCP server, and each surface calls its tools with the same key.

Publish gpt-6.1-sol for Copilot

Deploy gpt-6.1-sol in Foundry, publish it through the gateway under the alias gpt-6.1-sol with a token limit of 100,000 tokens per minute for each caller, and create the runtime key github-copilot. Pick a track.

Choose your track
1

Deploy the model in Foundry

In the Foundry portal, open the Foundry account behind your gateway and deploy gpt-6.1-sol with the deployment name gpt-6.1-sol. Or paste the command from the Azure CLI track's first step into Azure Cloud Shell.

2

Import the deployment

  1. Open the AI Gateway portal and select your gateway in the top bar.
  2. Select ModelsAddModels from Microsoft Foundry.
  3. In Subscriptions and Resource groups, select those of your Foundry account. Keep Managed identity as the Authentication method.
  4. In Models, select only the gpt-6.1-sol deployment: the page can preselect other deployments of the account.
  5. Under Aliases, keep Create an alias for each model selected, then select Add models.
3

Publish the gpt-6.1-sol alias

The import names its alias after the provider and the deployment, for example <foundry-account-name>/gpt-6.1-sol. Copilot needs the plain model name, so add an alias called gpt-6.1-sol:

  1. Select ModelsAddAlias for existing models.
  2. On Basics, set Alias name to gpt-6.1-sol. Select Next.
  3. On Select models, select the gpt-6.1-sol backend model in the grid. Select Next.
  4. Keep the defaults on Auto model selection (preview), Session affinity, Priorities and weights, and Failure handling, selecting Next on each, then select Create on Review.
4

Add the token limit

  1. Select PoliciesAdd policy. On the Type step, under Cost and rate limits, select Token rate limit, then Next.
  2. On the Scope step, select Individual assets, select the gpt-6.1-sol asset of type Alias, then Next.
  3. On the Details step, set Time period to Per minute and Allowed tokens to 100000. Keep Counter dimensions at Identity, so that each key gets its own counter. Select Create.
5

Create the key for Copilot

  1. Select AccessAPI keysCreate API key.
  2. In Key name, enter github-copilot and select Create.
  3. Copy the value from API key created successfully, for example with Copy & done, and paste it into your shell without echoing it:
# Paste the key without showing it, and export it for the test calls and the Copilot surfaces
printf "AI Gateway API key: " && read -rs AI_GATEWAY_API_KEY && export AI_GATEWAY_API_KEY && echo
echo "🔑 Key: in AI_GATEWAY_API_KEY (${#AI_GATEWAY_API_KEY} characters, not shown)"
Checkpoint

ModelsAliases lists gpt-6.1-sol, Policies lists a Token rate limit of 100,000 tokens per minute scoped to it, and AccessAPI keys lists github-copilot.

Test the endpoint

Before you configure Copilot, call the alias the way Copilot does: the Responses API with a bearer token.

# Call the alias the way Copilot does: the Responses API, with the key as a bearer token
curl -sS "${AI_GATEWAY_URL%/}/default/models/openai/v1/responses" \
  -H "Authorization: Bearer $AI_GATEWAY_API_KEY" -H "Content-Type: application/json" \
  -d "{\"model\":\"$AI_GATEWAY_MODEL\",\"input\":\"In one sentence, what is a code review for?\"}" \
  | jq -r 'if .output then "✨ Model: \(.model)",
      "🤖 \([.output[] | select(.type == "message") | .content[] | select(.type == "output_text") | .text] | join(""))"
    else "🚨 \(.error.message // .)" end'

Expected output (your text differs):

Text
✨ Model: gpt-6.1-sol
🤖 A code review checks changes for bugs, clarity, and consistency before they're merged.
Note

A new alias, policy, or key can take a minute to apply. If the call returns 401 or 404, wait a minute and try again.

Connect VS Code

VS Code adds your own models through a Custom Endpoint model group. It stores the key in its secret storage and the rest in chatLanguageModels.json.

1

Add the gateway as a custom endpoint

  1. Open the practice project in VS Code, for example with code ~/contoso-calc.

  2. Open the Command Palette (Ctrl+Shift+P, or Cmd+Shift+P on macOS) and run Chat: Manage Language Models. The Language Models editor opens.

    Chat: Manage Language Models opens the Language Models editor. Add Models starts the custom endpoint setup.
  3. Select Add ModelsCustom Endpoint, and enter AI Gateway as the group name.

  4. Paste your github-copilot key into AI Gateway: API Key and press Enter. To copy it from the shell without printing it, run printf %s "$AI_GATEWAY_API_KEY" | pbcopy on macOS, or Set-Clipboard $env:AI_GATEWAY_API_KEY in PowerShell.

    VS Code masks the key and keeps it in its secret storage, not in a settings file.
  5. In AI Gateway: API Type, select Responses.

    Pick Responses. It becomes the default for every model in the group.
2

Add gpt-6.1-sol to the group

VS Code opens chatLanguageModels.json with the new group. The gateway has no model list to discover, so add the model to models yourself. Print the URL to paste:

Bash
# Print the full Responses URL on your gateway, to paste into chatLanguageModels.json
echo "🌐 Responses URL: ${AI_GATEWAY_URL%/}/default/models/openai/v1/responses"

Then make the group look like this. Keep the apiKey value that VS Code wrote: it points at the secret, not the key itself.

chatLanguageModels.json
[
  {
    "name": "AI Gateway",
    "vendor": "customendpoint",
    "apiKey": "${input:chat.lm.secret.<id>}",
    "apiType": "responses",
    "models": [
      {
        "id": "gpt-6.1-sol",
        "name": "GPT-6.1 Sol (AI Gateway)",
        "url": "https://<gateway-host>/default/models/openai/v1/responses",
        "toolCalling": true,
        "vision": true,
        "maxInputTokens": 128000,
        "maxOutputTokens": 16000
      }
    ]
  }
]
The id is the alias that VS Code sends as the model. The name is what the model picker shows.
FieldWhat it does
idThe alias, sent as model in each request.
nameThe label in the model picker. Copilot already offers a built-in GPT-6.1 Sol, so a distinct name tells them apart.
urlThe full Responses URL on your gateway.
toolCallingTells VS Code that the model can call tools, which agent mode needs.
maxInputTokens, maxOutputTokensHow much context VS Code sends and how long a reply it asks for.

Save the file.

3

Pick the model and run an agent task

  1. Open the Chat view (Ctrl+Alt+I, or Ctrl+Cmd+I on macOS) and select Agent.

  2. Open the model picker, select the AI Gateway tab, and select GPT-6.1 Sol (AI Gateway). If the model isn't listed, run Developer: Reload Window.

    Models from the custom endpoint get their own tab in the model picker.
  3. Send this prompt:

    Text
    Fix the bug in calc.py and add a guard for division by zero

Copilot reads the file, changes add to return a + b, makes divide raise ZeroDivisionError when b is zero, and checks its work. Review the diff and select Keep.

Agent mode on gpt-6.1-sol through the gateway: the diff in the editor, the summary in chat, and Keep or Undo for the change.
Checkpoint

The chat input shows GPT-6.1 Sol (AI Gateway), and calc.py now adds correctly and refuses to divide by zero.

Tip

Without a GitHub sign-in, VS Code asks you to pick utility models for chat titles, commit messages, and pull request descriptions. Set chat.utilityModel and chat.utilitySmallModel to your gateway model. Inline suggestions still need a GitHub sign-in.

Connect the Copilot CLI

The Copilot CLI reads its provider from environment variables, so you can switch between the gateway and GitHub's models per terminal.

1

Point the CLI at the gateway

VariableValue
COPILOT_PROVIDER_BASE_URLhttps://<gateway-host>/default/models/openai/v1. Setting it turns on BYOK.
COPILOT_PROVIDER_WIRE_APIresponses. The default is completions.
COPILOT_PROVIDER_API_KEYYour github-copilot key, sent as a bearer token.
COPILOT_MODELgpt-6.1-sol, the alias.
COPILOT_OFFLINEOptional. true also turns off the GitHub sign-in, telemetry, web tools, and the GitHub MCP server.
# Point the Copilot CLI at the gateway's OpenAI-compatible base URL, over the Responses API
export COPILOT_PROVIDER_BASE_URL="${AI_GATEWAY_URL%/}/default/models/openai/v1"
export COPILOT_PROVIDER_WIRE_API="responses"

# Give the CLI the github-copilot key, which it sends as a bearer token, and the alias as its model
export COPILOT_PROVIDER_API_KEY="$AI_GATEWAY_API_KEY"
export COPILOT_MODEL="$AI_GATEWAY_MODEL"
echo "📝 Copilot CLI: $COPILOT_MODEL over the $COPILOT_PROVIDER_WIRE_API API"
echo "🌐 Base URL: $COPILOT_PROVIDER_BASE_URL"
echo "🔑 Key: in COPILOT_PROVIDER_API_KEY (${#COPILOT_PROVIDER_API_KEY} characters, not shown)"

These variables last for the current terminal. Run copilot help environment for the full list.

2

Ask a quick question

copilot -p runs one prompt and exits:

Bash
# Move into the practice project
cd ~/contoso-calc &&
  echo "📂 Practice project: ~/contoso-calc"

# Run one prompt on the gateway's model, then exit
echo "🤖 Copilot CLI answer:"
copilot -p "Which model are you? Answer in one short sentence."

Expected output (your numbers differ):

Text
📂 Practice project: ~/contoso-calc
🤖 Copilot CLI answer:
I'm powered by gpt-6.1-sol (model ID: gpt-6.1-sol).

Changes    +0 -0
Duration   6s
Tokens     ↑ 14.4k (14.4k written) • ↓ 24
Resume     copilot --resume=<session-id>

The Tokens line shows how much context even a short question carries. Resume reopens the session in interactive mode.

3

Run an agent task

Start an interactive session in the project, and select Yes when the CLI asks whether you trust the folder:

Bash
# Start an interactive Copilot CLI session in the practice project, on the gateway's model
echo "🚀 Starting the Copilot CLI on $COPILOT_MODEL (type /exit to leave)"
copilot

Send this prompt:

Text
Add a multiply function to calc.py and unittest tests in test_calc.py, then run them

Copilot plans the change and asks before it edits a file or runs a command. Select Yes to approve each step, or Yes, and don't ask again to approve that kind of step for this folder.

Before each edit, the CLI shows the diff and asks for approval.
The task is done: the CLI added multiply, wrote the tests, and ran them. The status bar shows gpt-6.1-sol.
Checkpoint

The status bar shows gpt-6.1-sol, and the tests pass. If you skipped the VS Code section, Copilot may also fix add when the first test run fails. Type /exit to leave the session.

Connect the Copilot SDK

The GitHub Copilot SDK runs the same agent as the CLI inside your own app. Pass a provider when you create a session, and the session sends its model calls to the gateway. The SDK packages bring the Copilot runtime with them, so you don't need to install the CLI. No GitHub sign-in is needed.

1

Review the samples

The samples read AI_GATEWAY_URL, AI_GATEWAY_API_KEY, and, optionally, AI_GATEWAY_MODEL from the environment, then ask the agent which model it is and to list the files in the current folder. They also read AI_GATEWAY_MCP_URL, which you set later in Add the Learn tools to the Copilot SDK. approveAll lets the agent run tools without asking, which suits a sample. In an app, decide which tools to allow in onPermissionRequest.

infra/copilot-sdk.tsGitHub
// Runs a GitHub Copilot SDK agent session on gpt-6.1-sol published by Azure AI Gateway (bring your own key).
// Set AI_GATEWAY_URL and AI_GATEWAY_API_KEY first. No GitHub sign-in is needed: every model call goes to the gateway.
// Set AI_GATEWAY_MCP_URL too, and the agent also gets the tools of an MCP server that the gateway publishes.
import { CopilotClient, approveAll, type MCPServerConfig } from "@github/copilot-sdk";

const gatewayUrl = process.env.AI_GATEWAY_URL?.replace(/\/$/, "");
const apiKey = process.env.AI_GATEWAY_API_KEY;
const model = process.env.AI_GATEWAY_MODEL ?? "gpt-6.1-sol";
const mcpUrl = process.env.AI_GATEWAY_MCP_URL;
if (!gatewayUrl || !apiKey) {
  throw new Error("Set AI_GATEWAY_URL and AI_GATEWAY_API_KEY first.");
}

// The MCP server gets the key as a bearer token, like the model calls.
const mcpServers: Record<string, MCPServerConfig> = mcpUrl
  ? { "ai-gateway-learn": { type: "http", url: mcpUrl, headers: { Authorization: `Bearer ${apiKey}` }, tools: ["*"] } }
  : {};

const client = new CopilotClient();
try {
  const session = await client.createSession({
    model,
    provider: {
      type: "openai",
      baseUrl: `${gatewayUrl}/default/models/openai/v1`,
      apiKey,
      wireApi: "responses",
    },
    mcpServers,
    onPermissionRequest: approveAll,
  });

  const prompt = process.argv.slice(2).join(" ") || "Which model are you? Then list the files in this folder.";
  const reply = await session.sendAndWait({ prompt });
  console.log(reply?.data.content);
  await session.disconnect();
} finally {
  await client.stop();
}
infra/copilot_sdk.pyGitHub
"""Runs a GitHub Copilot SDK agent session on gpt-6.1-sol published by Azure AI Gateway (bring your own key).

Set AI_GATEWAY_URL and AI_GATEWAY_API_KEY first. No GitHub sign-in is needed: every model call goes to the gateway.
Set AI_GATEWAY_MCP_URL too, and the agent also gets the tools of an MCP server that the gateway publishes.
"""

import asyncio
import os
import sys

from copilot import CopilotClient, PermissionHandler


async def main() -> None:
    gateway_url = os.environ["AI_GATEWAY_URL"].rstrip("/")
    api_key = os.environ["AI_GATEWAY_API_KEY"]
    model = os.environ.get("AI_GATEWAY_MODEL", "gpt-6.1-sol")
    mcp_url = os.environ.get("AI_GATEWAY_MCP_URL")

    # The MCP server gets the key as a bearer token, like the model calls.
    headers = {"Authorization": f"Bearer {api_key}"}
    mcp_servers = (
        {"ai-gateway-learn": {"type": "http", "url": mcp_url, "headers": headers, "tools": ["*"]}}
        if mcp_url
        else {}
    )

    client = CopilotClient()
    await client.start()
    try:
        session = await client.create_session(
            model=model,
            provider={
                "type": "openai",
                "base_url": f"{gateway_url}/default/models/openai/v1",
                "api_key": api_key,
                "wire_api": "responses",
            },
            mcp_servers=mcp_servers,
            on_permission_request=PermissionHandler.approve_all,
        )
        prompt = " ".join(sys.argv[1:]) or "Which model are you? Then list the files in this folder."
        reply = await session.send_and_wait(prompt, timeout=180)
        print(reply.data.content if reply else "(no reply)")
        await session.disconnect()
    finally:
        await client.stop()


if __name__ == "__main__":
    asyncio.run(main())
2

Run a sample

Run the commands from the lab folder (labs/24-github-copilot-byok).

# Install the samples' dependencies in the infra folder: the Copilot SDK, which brings the Copilot runtime with it
cd infra
npm install          # @github/copilot-sdk
echo "📦 The Copilot SDK for TypeScript is installed"

# Run the sample: the agent says which model it is, and lists the files in the folder
echo "🤖 Agent reply:"
npm start            # runs copilot-sdk.ts on Node.js 22.18 or later

Expected output (your text differs):

Text
🤖 Agent reply:
I'm powered by gpt-6.1-sol (model ID: gpt-6.1-sol).

Files in this folder:
- copilot-model.bicep
- copilot-sdk.ts
- copilot_sdk.py
- deployment.bicep
- main.bicep
- main.parameters.json
- package.json
- requirements.txt

The samples print the agent's last message. If the agent answers in two messages, you see only the file list.

The SDK session runs on gpt-6.1-sol through the gateway, and its tools work as in the CLI.

To send your own prompt, pass it as an argument, for example npm start -- "Summarize package.json" or python copilot_sdk.py "Summarize requirements.txt".

Checkpoint

The sample names gpt-6.1-sol and lists the folder's files.

Connect the GitHub Copilot app

The GitHub Copilot app is a desktop app for running Copilot agent sessions across your projects. It adds your own models as model providers and keeps their keys in the system credential store.

Note

The GitHub Copilot app is in public preview, and its screens change often. These steps follow Use BYOK models in the GitHub Copilot app. The picture below is an illustration, not a screenshot, so labels in your version can differ.

1

Add the gateway as a model provider

  1. Open the GitHub Copilot app and sign in with your GitHub account.

  2. Open the app settings and select Model providersAdd provider.

  3. Select Any OpenAI-compatible HTTP endpoint. Don't pick Azure OpenAI or Microsoft Foundry: they expect those services' own URLs and keys.

  4. Enter the provider details:

    FieldValue
    Display nameAI Gateway
    Base URLhttps://<gateway-host>/default/models/openai/v1
    API keyYour github-copilot key
    Wire API, if shownResponses
  5. The gateway doesn't serve a /models list, so the app can't discover its models. Add the model by hand with the model ID gpt-6.1-sol.

  6. Select Add provider.

2

Start a session on gpt-6.1-sol

The provider's models appear in the model picker next to GitHub's. Start a session on the contoso-calc folder, select gpt-6.1-sol from the AI Gateway provider, and ask a question, for example Which model are you, and what does calc.py do?.

Checkpoint

The reply names gpt-6.1-sol, and the request shows up in the gateway's Monitoring under the github-copilot key.

Publish Microsoft Learn tools for Copilot

Copilot's agents call tools as well as models. Publish an MCP server through the same gateway, and Copilot's tool calls get the same controls as its model calls: one key that you can suspend, one endpoint per tool server, and one place to decide which tools agents see. Here, the gateway publishes the public Microsoft Learn MCP server as learn, so Copilot can look up current Microsoft documentation while it codes on gpt-6.1-sol.

Each Copilot surface then has two connections to the gateway:

ModelMCP server
URLhttps://<gateway-host>/default/models/openai/v1https://<gateway-host>/default/toolservers/learn/mcp
How the key is sentAuthorization: Bearer <key>The same Authorization: Bearer <key> header, which you set yourself
Keygithub-copilotThe same github-copilot key. It needs no extra setup.
Note

Learn results are large: one search returns 25 to 50 KB of text. Copilot keeps big results out of the prompt where it can, but tasks that use the tools still send more tokens. In testing, a one-shot question that used one search sent about 51,000 input tokens, and the agent task in the CLI section sent about 150,000 over seven minutes, within the 100,000 tokens-per-minute limit.

Tip

If you did Federate MCP tools behind one endpoint, your gateway already has learn. Skip to Test the MCP endpoint.

Create the learn server with one backend, https://learn.microsoft.com/api/mcp, under the namespace learn. The Learn server is public, so the backend needs no credentials. Pick a track.

Choose your track
1

Register the Learn server

  1. Open the AI Gateway portal, select your gateway, and select MCP serversAdd MCP server.
  2. On Basic info, set Name to learn, Display name to Microsoft Learn, and Description to Search and fetch official Microsoft documentation and code samples. In Supported MCP versions, select 2025-06-18, the version that the Learn server reports. Select Next.
  3. On Backends, select Add backendMCP server. Set Endpoint URL to https://learn.microsoft.com/api/mcp, keep Authentication at None, and set Namespace to learn. Select Add, then Next, which saves the server and opens Tools.
  4. On Tools, keep Allow all tools, and select Create.

Register the Learn MCP server in lab 04 explains each field.

2

Copy the endpoint

When the server is ready, copy the Endpoint URL from its details page. It ends in /default/toolservers/learn/mcp.

Checkpoint

MCP servers lists learn with one backend in the namespace learn.

1

Test the MCP endpoint

Save the endpoint, then list its tools the way Copilot does: a JSON-RPC tools/list request with the key as a bearer token.

# Save the endpoint of the learn MCP server on your gateway
export AI_GATEWAY_MCP_URL="${AI_GATEWAY_URL%/}/default/toolservers/learn/mcp"
echo "🔌 MCP endpoint: $AI_GATEWAY_MCP_URL"

# List the server's tools the way Copilot does: a JSON-RPC tools/list request, with the key as a bearer token
echo "🔧 Tools on the learn MCP server:"
curl -sS "$AI_GATEWAY_MCP_URL" \
  -H "Authorization: Bearer $AI_GATEWAY_API_KEY" -H "Content-Type: application/json" \
  -H "Accept: application/json, text/event-stream" \
  -d '{"jsonrpc":"2.0","id":1,"method":"tools/list"}' | jq -r '.result.tools[].name'

Expected output:

Text
🔌 MCP endpoint: https://<gateway-host>/default/toolservers/learn/mcp
🔧 Tools on the learn MCP server:
learn_microsoft_docs_search
learn_microsoft_code_sample_search
learn_microsoft_docs_fetch

Each name starts with the backend's namespace, learn. The Learn server can add tools over time, and Allow all tools publishes them too.

Add the Learn tools to VS Code

VS Code reads MCP servers from .vscode/mcp.json in the workspace. An inputs entry makes VS Code ask for the key once and keep it in its secret storage, so the file holds no secret.

1

Add the server to the project

Print the endpoint to paste:

Bash
# Print the endpoint of the learn MCP server, to paste into .vscode/mcp.json
echo "🔌 MCP endpoint: $AI_GATEWAY_MCP_URL"

In VS Code, create .vscode/mcp.json in contoso-calc with this content, and replace the URL with yours:

.vscode/mcp.json
{
  "inputs": [
    {
      "type": "promptString",
      "id": "aigw-key",
      "description": "AI Gateway key (github-copilot)",
      "password": true
    }
  ],
  "servers": {
    "ai-gateway-learn": {
      "type": "http",
      "url": "https://<gateway-host>/default/toolservers/learn/mcp",
      "headers": {
        "Authorization": "Bearer ${input:aigw-key}"
      }
    }
  }
}
2

Start the server

  1. Select Start above ai-gateway-learn in mcp.json.

  2. Paste your github-copilot key into the AI Gateway key (github-copilot) prompt, and press Enter.

    VS Code asks for the key the first time the server starts, masks it, and keeps it in its secret storage.
  3. The line above the server changes to Running with 3 tools.

    The server is connected through the gateway and offers three tools. mcp.json shows the key only as a reference to the input.
3

Check the tools

Open the Chat view, select Agent and GPT-6.1 Sol (AI Gateway), and select Configure Tools (the tools icon in the chat input). ai-gateway-learn lists the three Learn tools. VS Code shows the name that the gateway reports for itself, Azure API Management, next to the server name.

The three tools come from the gateway, not from learn.microsoft.com directly.
4

Run a task with the tools

Send this prompt:

Text
Use the Microsoft Learn tools to look up the Azure Functions Python v2 programming model, then add function_app.py that exposes add() from calc.py as an HTTP-triggered function. Cite the Learn page you used.

VS Code asks before each tool call and shows the tool's input. Select Allow in this Session to trust the Learn tools for the rest of this chat.

Each tool call names the tool and its MCP server, and shows the input that Copilot sends through the gateway.

Copilot searches Learn, fetches the page it needs, writes function_app.py with the v2 decorators, and cites the page. Review the diff and select Keep.

Agent mode on gpt-6.1-sol used the gateway's Learn tools, then wrote the function and cited its source.
Checkpoint

function_app.py exposes add as an HTTP function, and the chat answer links to a Microsoft Learn page.

Add the Learn tools to the Copilot CLI

The Copilot CLI keeps MCP servers in ~/.copilot/mcp-config.json, and the GitHub Copilot app reads the same file. Header values can refer to environment variables, so the file doesn't need to hold the key.

1

Add the server

Keep the single quotes around the header. They stop the shell from expanding ${AI_GATEWAY_API_KEY}, so the file stores the reference, and the CLI reads the key from its environment each time it connects.

# Add the gateway's learn MCP server to the Copilot CLI, with a header that reads the key from the environment
copilot mcp add --transport http --header 'Authorization: Bearer ${AI_GATEWAY_API_KEY}' ai-gateway-learn \
  "$AI_GATEWAY_MCP_URL" &&
  echo "🔌 MCP server ai-gateway-learn is ready in the Copilot CLI"

# List the MCP servers that the Copilot CLI uses
echo "🔌 MCP servers in the Copilot CLI:"
copilot mcp list

Expected output (your built-in servers can differ):

Text
Added server "ai-gateway-learn"

ai-gateway-learn
  Type: http
  URL: https://<gateway-host>/default/toolservers/learn/mcp
  Headers:
    Authorization: ***
  Tools: * (all)
  Source: User
🔌 MCP server ai-gateway-learn is ready in the Copilot CLI
🔌 MCP servers in the Copilot CLI:
User servers:
  ai-gateway-learn (http)

Builtin servers:
  computer-use (local)
  github-mcp-server (http)
The CLI masks header values. Run copilot mcp get ai-gateway-learn to see the server again.

The file now holds the reference, not the key:

~/.copilot/mcp-config.json
{
  "mcpServers": {
    "ai-gateway-learn": {
      "tools": ["*"],
      "type": "http",
      "url": "https://<gateway-host>/default/toolservers/learn/mcp",
      "headers": {
        "Authorization": "Bearer ${AI_GATEWAY_API_KEY}"
      }
    }
  }
}
2

Check the tools in a session

The copilot session needs the BYOK variables from Connect the Copilot CLI and AI_GATEWAY_API_KEY. If you did the VS Code section, remove its result first, so the CLI starts from the same files:

Bash
# Move into the practice project, and remove the files that the VS Code section added, if any
cd ~/contoso-calc && rm -f function_app.py host.json requirements.txt &&
  echo "📂 Practice project is ready: ~/contoso-calc"

# Start an interactive Copilot CLI session, on the gateway's model and with the gateway's Learn tools
echo "🚀 Starting the Copilot CLI with the Learn tools (type /exit to leave)"
copilot

The CLI reports MCP Servers reloaded: 1 server connected. Run /mcp show ai-gateway-learn to list the tools, then press Esc twice to close the list.

/mcp show lists the server's tools with their descriptions.
3

Run a task with the tools

Send the same prompt as in VS Code:

Text
Use the Microsoft Learn tools to look up the Azure Functions Python v2 programming model, then add function_app.py that exposes add() from calc.py as an HTTP-triggered function. Cite the Learn page you used.

The CLI asks before it uses each tool. Select Yes, and don't ask again for the Learn tools that you trust. The tool lines show (MCP: ai-gateway-learn).

Learn results are large, so the CLI saves them to a file and reads the parts it needs with shell commands, which it also asks you to approve.

Copilot writes function_app.py, and may add requirements.txt and host.json, then summarizes the change and cites the Learn page.

The model and the tools both went through the gateway with the github-copilot key.

Type /exit to leave the session.

4

Use the tools in one-shot mode

copilot -p can't ask for approval, so allow the server's tools up front with --allow-tool:

Bash
# Run one prompt that needs the Learn tools, and allow the server's tools up front, because -p can't ask
echo "🤖 Copilot CLI answer, with the Learn tools:"
copilot -p "Search Microsoft Learn: how do I run a Python function app locally with Azure Functions Core Tools? Answer in two sentences with a link." --allow-tool='ai-gateway-learn'

Expected output (shortened, your text differs):

Text
🤖 Copilot CLI answer, with the Learn tools:
● learn_microsoft_docs_search (MCP: ai-gateway-learn) · run Python Azure Functions app locally Core…
  └ Output too large to read at once (52.0 KB). Saved to: /var/folders/…

Install Azure Functions Core Tools v4 and a supported Python version, activate your project's virtual environment,
and install dependencies with `pip install -r requirements.txt`. From the project root, run `func start`, then open
the HTTP function URL printed in the terminal; see [Microsoft Learn](https://learn.microsoft.com/azure/azure-functions/functions-run-local#start-the-functions-runtime).

Changes    +0 -0
Duration   18s
Tokens     ↑ 50.8k (30.4k cached, 20.4k written) • ↓ 204
Resume     copilot --resume=<session-id>

Compare the Tokens line with the 14,400 tokens of the plain question in Ask a quick question: the search result made up most of the difference.

Checkpoint

/mcp show lists three tools, the tool lines show (MCP: ai-gateway-learn), and the answer cites a Learn page.

Tip

COPILOT_OFFLINE=true turns off the built-in GitHub MCP server, but not the servers you add: the Learn tools keep working.

Add the Learn tools to the Copilot SDK

The samples already handle MCP. When AI_GATEWAY_MCP_URL is set, they add the server to the session as ai-gateway-learn, with the key as a bearer token in the Authorization header and tools: ["*"] to offer every tool:

copilot-sdk.ts
const mcpServers: Record<string, MCPServerConfig> = mcpUrl
  ? { "ai-gateway-learn": { type: "http", url: mcpUrl, headers: { Authorization: `Bearer ${apiKey}` }, tools: ["*"] } }
  : {};

The session gets them with mcpServers in TypeScript and mcp_servers= in Python. approveAll lets the agent call the tools without asking.

1

Run a sample with the tools

Run the commands from labs/24-github-copilot-byok/infra, in the shell where you set AI_GATEWAY_MCP_URL.

# Run copilot-sdk.ts with a prompt that asks the agent to use the gateway's Learn tools
echo "🤖 Agent reply, with the Learn tools:"
npm start -- "Use the Microsoft Learn tools to answer: how do I run a Python function app locally with Azure Functions Core Tools? Answer in two sentences and cite the Learn page."

Expected output (your text differs):

Text
🤖 Agent reply, with the Learn tools:
Install Azure Functions Core Tools, activate your Python virtual environment, install your app's dependencies, and
run `func start` from the project root, as described in [Microsoft Learn](https://learn.microsoft.com/en-us/azure/azure-functions/functions-run-local?pivots=programming-language-python#start-the-functions-runtime).
For an HTTP-triggered function, open the localhost URL printed in the terminal (usually on port 7071) or call it
with an HTTP client.
Checkpoint

The answer cites a learn.microsoft.com page that the agent found through the gateway.

Add the Learn tools to the GitHub Copilot app

The GitHub Copilot app loads the MCP servers that you configured for the Copilot CLI. If you did the CLI section, ai-gateway-learn is already there.

1

Give the app the key

The server's header reads ${AI_GATEWAY_API_KEY} from the app's environment, not from your terminal. Make the variable available to the app, then quit and reopen it. For example, export it in your shell profile on macOS or Linux, or set it as a user environment variable on Windows. Prefer a command that reads the key from a secret store over the key itself in plain text.

If you'd rather not set a variable, replace ${AI_GATEWAY_API_KEY} in ~/.copilot/mcp-config.json with the key. The file then holds the key in plain text, so keep it private.

2

Check the server

Select Customize in the app sidebar, then MCPInstalled. The list includes ai-gateway-learn. If you skipped the CLI section, run its copilot mcp add command, or add a custom server here with the endpoint URL and an Authorization header set to Bearer <key>.

3

Run a task with the tools

Sessions load MCP servers when they start, so start a new session on the contoso-calc folder. Select gpt-6.1-sol from the AI Gateway provider, and send:

Text
Use the Microsoft Learn tools to find how to run calc.py as an Azure Functions app locally, and summarize the steps with a link.

The session's agent sees each tool with the server name in front, such as ai-gateway-learn-learn_microsoft_docs_search.

Checkpoint

The reply cites a Microsoft Learn page, and the session shows calls to ai-gateway-learn tools.

Note

The GitHub Copilot app is in public preview, and its screens change often. These steps follow Customizing the GitHub Copilot app.

Verify

Every surface you connected sends its requests with the github-copilot key. Use it to see the traffic and to switch Copilot off.

1

See Copilot's usage by key

If you set up Monitoring in Monitor usage, cost, and traces, open MonitoringModels in the AI Gateway portal. gpt-6.1-sol lists the calls and the input and output tokens from your Copilot sessions, and Model usage by caller identity shows them under the github-copilot key. The input tokens are far higher than the output tokens: that's the context that the agent sends with every request.

2

Cut Copilot off by suspending the key

Suspend the key, then run the CLI again. The settings are unchanged, but the gateway now rejects the key.

# Suspend the github-copilot key: calls with it get 401 until you set it back to active
STATE="$(az aigateway api-key update -g "$RG" --gateway-name "$GATEWAY" -n "$KEY_NAME" --state suspended \
  --query properties.state -o tsv)" &&
  echo "🔑 API key github-copilot is $STATE"

# Give the gateway time to apply the change
echo "⏳ Waiting 30 seconds for the key change to apply"
sleep 30

# Ask the Copilot CLI again, with the same settings: the gateway now rejects the key
echo "🚦 Expect HTTP 401 from the gateway, after two retries:"
cd ~/contoso-calc && copilot -p "Say hi"

Expected output, after the CLI retries twice:

Text
🔑 API key github-copilot is suspended
⏳ Waiting 30 seconds for the key change to apply
🚦 Expect HTTP 401 from the gateway, after two retries:
Request failed due to a transient API error. Retrying...
Request failed due to a transient API error. Retrying...
Authentication failed with provider at https://<gateway-host>/default/models/openai/v1 (HTTP 401).
  Check your COPILOT_PROVIDER_API_KEY, COPILOT_PROVIDER_API_KEY_COMMAND, or COPILOT_PROVIDER_BEARER_TOKEN.

VS Code, the SDK, and the app get the same 401. One key change cuts off every Copilot surface that uses the key, on every machine, without touching their settings.

The key guards the tools too. While it's suspended, the MCP endpoint also answers 401 Unauthorized: run the request from Test the MCP endpoint without | jq … to see it, and the Learn tools fail to connect in every surface.

3

Turn Copilot back on

# Turn the github-copilot key back on
STATE="$(az aigateway api-key update -g "$RG" --gateway-name "$GATEWAY" -n "$KEY_NAME" --state active \
  --query properties.state -o tsv)" &&
  echo "🔑 API key github-copilot is $STATE"

# Give the gateway time to apply the change
echo "⏳ Waiting 30 seconds for the key change to apply"
sleep 30

# Ask the Copilot CLI again: it answers through the gateway
echo "🤖 Copilot CLI answer:"
copilot -p "Say hi in three words"
Checkpoint

The CLI failed with HTTP 401 while the key was suspended, and answered again after 🔑 API key github-copilot is active.

Go further

  • Choose which Learn tools Copilot sees. Publish only learn_microsoft_docs_search, block a tool, or cap tool calls per caller in Control which tools agents can call. Copilot picks up the change the next time it lists tools, with no change on the clients.
  • Give Copilot more tools from one endpoint. Add more backends to the server, such as an OpenAPI spec or a Connector Namespace server imported from Azure, as in Federate MCP tools behind one endpoint. Each backend's namespace keeps the tool names apart.
  • Sign in with Microsoft Entra ID instead of a key. If the gateway accepts Entra tokens, as in Authorize callers with Microsoft Entra ID, set COPILOT_PROVIDER_BEARER_TOKEN for the CLI, or pass a bearer token in the SDK's provider settings. Tokens expire, so refresh them for long sessions.
  • Fetch the key when it's needed. COPILOT_PROVIDER_API_KEY_COMMAND runs a command for each request and uses its output as the key, for example az keyvault secret show --vault-name <vault> -n github-copilot --query value -o tsv. The key then never sits in your shell profile, and a rotated key takes effect right away.
  • Give each developer or tool its own key. Keys are cheap, and per-key usage in Monitoring shows who uses how much. Rotate or revoke them with Manage runtime API keys.
  • Route Copilot across models. Point the gpt-6.1-sol alias at several backend models, as in Route by priority and weight. VS Code chains Responses API turns with previous_response_id by default; if the backends don't share state, set "statefulResponses": false on the model in chatLanguageModels.json.
  • Offer more models. Publish more aliases and add each one to models in VS Code, or to the provider in the app. For the CLI, set COPILOT_MODEL to a different alias. If an alias doesn't have a well-known model name, set COPILOT_PROVIDER_MODEL_ID to the model it serves, so the CLI picks the right prompts and token sizes.

Clean up

Remove the gateway from each Copilot surface you configured, and delete the practice project:

  • VS Code: delete the AI Gateway group from chatLanguageModels.json, or remove it in the Language Models editor. Delete .vscode/mcp.json from contoso-calc.
  • Copilot CLI: run copilot mcp remove ai-gateway-learn. Then close the terminal, or run unset COPILOT_PROVIDER_BASE_URL COPILOT_PROVIDER_WIRE_API COPILOT_PROVIDER_API_KEY COPILOT_MODEL (Bash) or Remove-Item Env:COPILOT_PROVIDER_*, Env:COPILOT_MODEL (PowerShell).
  • GitHub Copilot app: remove the AI Gateway provider in Model providers. copilot mcp remove also removes ai-gateway-learn from the app. Remove AI_GATEWAY_API_KEY from your profile or environment if you set it for the app.
  • Practice project: run rm -rf ~/contoso-calc (Bash) or Remove-Item -Recurse -Force ~/contoso-calc (PowerShell).

Then delete what you created on the gateway. Pick a track.

Choose your track
1

Delete the key, the model, and the Learn server

  1. Delete github-copilot in AccessAPI keys.
  2. If you don't need gpt-6.1-sol any more, delete its alias and its backend model in Models, then its deployment in Foundry. If you imported the model in the portal, also delete its <foundry-account-name>/gpt-6.1-sol alias. Keep the gateway and the provider: the other labs reuse them.
  3. Keep the learn MCP server if you continue with Control which tools agents can call, which builds on it. Otherwise, delete it in MCP servers.

azd down and terraform destroy don't touch the gateway, provider, and key from lab 02, because the Bicep + azd and Terraform tracks created their own gateways.

Next steps

Finished the lab?

Mark it complete to track your progress across paths. It's saved only in this browser.