Azure AI Gateway Labs
Documentation Open portal
Observability & FinOps ยท Lab 14New

Attribute tokens and cost with OpenTelemetry

Export Microsoft Entra token claims with the gateway's logs, traces, and token metrics, then turn observed tokens into a per-tenant, per-app, and per-key chargeback report with KQL, PromQL, and a small Python script.

Advanced 35 min PortalCLIBicepTerraform Updated
Source Edit

What you'll do

  • Choose which JWT claims the gateway exports to logs, root traces, and token and cost metrics
  • Check which claims your callers' tokens actually carry before you rely on them
  • Build a chargeback report from request logs by tenant, client app, and API key, with failed requests and the gateway's cost estimate
  • Query token and cost metrics with PromQL without double counting token types

Before you start

Tracks: Portal, Azure CLI, Bicep + azd, Terraform

Three teams and a partner tenant share your gateway. Some call it with API keys, others with Microsoft Entra ID. Finance wants a monthly report of who used which model and what it cost. In this lab you export the right token claims with the gateway's telemetry, send traffic as both kinds of caller, and build that report from the data Application Insights already collects.

Before you begin

One-time setupThe Setup guide tasks that this lab uses. Mark each one done once, and the labs stop listing it.

Tip

On the Bicep + azd and Terraform tracks, azd up or terraform apply creates its own gateway, model, key, Application Insights, and role assignment, so you can skip this section. You need only the one-time setup above, and permission to create role assignments, for example Owner on the subscription. The later sections also use Python 3.10 or later.

  • Complete Connect Application Insights in lab 13, so the gateway exports all three signals to Application Insights with an OpenTelemetry exporter.
  • Complete Go keyless with Microsoft Entra ID and RBAC, or make sure your signed-in user has AI Gateway Models User (or AI Gateway User) on the gateway or on the model alias.
  • Permission to change the gateway, for example Contributor on its resource group, and Log Analytics Reader on the workspace behind Application Insights.
  • Python 3.10 or later.
  • Run the scripts and samples from the lab's folder, ~/aigateway/labs/14-otel-chargeback.

Set these variables once. If you named the Application Insights resource differently in lab 13, change $GATEWAY-insights. The portal's setup adds a short random suffix when that name is taken, such as <gateway>-insights-ak7s: list your components with az resource list -g "$RG" --resource-type Microsoft.Insights/components --query "[].name" -o tsv.

# Name your gateway and the alias to call
export RG="<resource-group>"
export GATEWAY="<gateway>"
export AI_GATEWAY_MODEL="<model-name>"     # a callable alias
echo "๐Ÿ“ Gateway $GATEWAY in $RG, alias $AI_GATEWAY_MODEL"

# Read the gateway's runtime URL
export AI_GATEWAY_URL="https://$(az aigateway show -g "$RG" -n "$GATEWAY" --query properties.frontend.defaultHostname -o tsv)"
echo "๐ŸŒ Runtime URL: $AI_GATEWAY_URL"

# Read your API key without showing it, and export it for the scripts
printf "AI Gateway API key: " && read -rs AI_GATEWAY_API_KEY && export AI_GATEWAY_API_KEY && echo
echo "๐Ÿ”‘ Key: in AI_GATEWAY_API_KEY (${#AI_GATEWAY_API_KEY} characters, not shown)"

# Find the Application Insights resource from lab 13, and the Log Analytics workspace behind it
APPI_ID="$(az resource show -g "$RG" -n "$GATEWAY-insights" --resource-type Microsoft.Insights/components --query id -o tsv)"
LAW_ID="$(az resource show --ids "$APPI_ID" --api-version 2020-02-02 --query properties.WorkspaceResourceId -o tsv)"
echo "๐Ÿ†” Application Insights: $APPI_ID"
echo "๐Ÿ†” Log Analytics workspace: $LAW_ID"

How it works

Every request already carries its caller. An API key call is attributed by the key's ID (azure.ai_gateway.api_key_id). A Microsoft Entra call carries a validated JWT, and token claim attributes copy the claims you pick into telemetry as azure.ai_gateway.identity.claims.<claim>. For each claim you choose the signals independently: logs, root traces, and observed token and cost metrics.

Logs hold the per-request token usage the backend reported, so they're the best source for a detailed report. Traces add the failures that HTTP status codes miss: a span gets the status Error when a model stream or an MCP call fails, even after HTTP 200. Metrics are cheaper to query over long ranges, but every exported claim value becomes a label, so keep metrics to claims with a bounded number of values.

Estimated cost is observed tokens ร— public pricing. The gateway exports its estimate with each request as azure.ai_gateway.client.token.cost, also for streamed responses whose logs report no token usage. Treat it as an allocation key and reconcile the total with provider billing or Azure Cost Management.

ClaimIdentifiesRecommended presetUse it for
tidThe caller's tenantLogs, Traces, MetricsTenant or customer totals. With oid, the dashboard's Entra caller.
oidThe user, service principal, or managed identityLogs, Traces, MetricsPer-caller totals
appid (v1 tokens), azp (v2 tokens)The client applicationLogs, TracesPer-app chargeback. This lab adds Metrics.
A custom claim, such as cost_centerWhatever your tokens carryYou chooseTeam chargeback, if the claim is in the token

Choose the claims to export

You keep the recommended preset and add Metrics for appid and azp, so token and cost metrics can also be split by client app.

Choose your track
1

Open the token claim attributes

In the AI Gateway portal, select your gateway, then MonitoringSettings. On the Logging tab, find Token claim attributes: "Add selected JWT claims to exported logs, root traces, and observed token and cost metrics." Each row is a claim with Logs, Traces, and Metrics checkboxes.

The recommended preset: tid and oid go to logs, traces, and metrics; appid and azp go to logs and traces.
Note

No Token claim attributes section? It's only shown for OpenTelemetry exporters. Reconfigure monitoring as in Monitor usage, cost, and traces. If the table is empty ("No token claims are included in exported telemetry"), add tid, oid, appid, and azp with Add claim.

2

Add metrics for the client app

Select Metrics for appid and for azp. Leave tid and oid on all three signals: the dashboard needs both on metrics to attribute Entra callers. Changes save automatically while "Saving changesโ€ฆ" is shown. Every claim needs at least one signal.

Tip

Add claim adds a row (the placeholder suggests cost_center) that starts with Logs and Traces. Only add claims that your callers' tokens carry. The next section shows how to check.

Checkpoint

tid and oid export to logs, traces, and metrics, and appid and azp now export to all three as well.

Send attributed traffic

1

Check what your token carries

A claim only appears in telemetry if the validated token has it. Decode your own token locally and look:

# Get a token for AI Gateway, and decode its claims locally, including the custom cost_center claim
echo "๐Ÿ‘ค Claims in your token for AI Gateway:"
az account get-access-token --scope https://ai.gateway.azure.com/Data.All --query accessToken -o tsv \
  | python3 infra/show_claims.py cost_center

On GitHub, open infra/show_claims.py. The script prints aud, ver, tid, oid, appid, azp, and idtyp, plus any claim names you pass. Nothing leaves your machine.

Expected output: aud is AI Gateway's app ID, 18550318-e177-4205-a743-6816d6fcdc3f, ver is 2.0, tid and oid are GUIDs, and azp holds the Azure CLI's client ID (a v1 token would carry it in appid). cost_center is (not in token).

Important

Tokens for AI Gateway are issued for its app ID, 18550318-e177-4205-a743-6816d6fcdc3f. Before you add a custom claim such as cost_center to Token claim attributes, decode a real token from that caller and confirm the claim is there. If it isn't, map client apps (appid or azp) to cost centers in your report instead.

2

Call the gateway as two callers

Run the script for your shell. It sends five requests with your API key and five with your Entra token, each with an x-correlation-id header that you can match in your client's own logs. The gateway doesn't add that header to its telemetry, so in Monitoring you find the calls by caller and time.

# Send five requests with your API key and five with your Entra token, and print the status of each one
echo "๐Ÿ“Š Requests by caller:"
./infra/send-attributed-traffic.sh "$AI_GATEWAY_MODEL" 5

On GitHub, open send-attributed-traffic.sh or send-attributed-traffic.ps1.

Expected output: ten rows, all 200, alternating between key and entra. A 401 or 403 on the entra rows means your role assignment is missing or hasn't propagated yet.

3

Find your callers in Monitoring

Wait a few minutes, then open MonitoringModels with 1H selected. In Model usage by caller identity, your API key and your Entra identity appear as separate rows with their own estimated cost, tokens, and calls: app-first-call as an API key, and your name as a User with your tenant. Entra callers are resolved from tid and oid. Callers with neither an API key nor those claims show as Unattributed.

Then select Traces. The Caller identities column shows the key or your name. Open one of your Entra requests: under Correlated logs, its log shows azure.ai_gateway.identity.claims.tid, ...oid, and ...appid or ...azp. An API key request shows azure.ai_gateway.api_key_id instead.

Checkpoint

Both callers appear on the Models tab, and an Entra request's correlated log shows its claims.

Build a chargeback report

The report groups the request logs by owner: requests, failed requests, token usage, and the gateway's own cost estimate. It then prices the tokens with a price list you control. The query is in infra/chargeback.kql:

infra/chargeback.kqlGitHub
// Chargeback starter: requests, tokens, and cost per tenant, caller, client app, API key, and model, from AI Gateway
// request logs. This is a starting point. Check the attribute names in one request's correlated log before you rely
// on a column.
//
// Where to run it: the Log Analytics workspace behind your Application Insights resource, Logs, KQL mode.
// Then export the result as CSV, or call the query API, and price it with chargeback_report.py.
//
// What it needs:
// - The Logs signal, and each claim you group by enabled for Logs in Monitoring โ€บ Settings โ€บ Logging โ€บ
//   Token claim attributes. Claims are exported as azure.ai_gateway.identity.claims.<claim>.
// - Token usage that the backend reported, exported as azure.ai_gateway.client.usage.*. Chat Completions reports
//   prompt_tokens and completion_tokens, and cached input tokens are part of the prompt tokens. The input_tokens and
//   output_tokens fallbacks cover APIs that use those names. Missing usage means "unknown", not zero: streamed
//   responses and failed calls often have none, and RequestsWithoutUsage counts them.
// - The gateway's own estimate, azure.ai_gateway.client.token.cost: observed tokens x public prices, in
//   azure.ai_gateway.currency. It's also there for streamed responses, so GatewayCost can exceed your token totals.
// - The Traces signal, for FailedRequests: a request failed if it answered HTTP 4xx or 5xx, or if one of its own
//   spans has the status Error, such as a model stream that reported an error after HTTP 200.
// Calls to an alias that doesn't exist have no model in their log, so they aren't counted.
//
// One request, one row before the totals:
// - A request's log has the span ID of its request span, the gateway's Server span. A client that sends a
//   traceparent header puts all its requests in one trace, so an Error span counts only for the request span above
//   it: HTTP POST โ€บ chat <model> โ€บ backend โ€บ request span.
// - A request that the gateway retried or failed over logs one record per attempt, with the same span ID and
//   azure.ai_gateway.routing.attempt 1, 2, and so on. The last attempt decides whether the request failed, and
//   the tokens and cost of every attempt add up.
let lookback = 30d;
let spans = OTelSpans
    | where TimeGenerated > ago(lookback)
    | project TraceId, SpanId, ParentSpanId, StatusCode, Attempt = toint(Attributes["azure.ai_gateway.routing.attempt"]);
let errorSpans = spans | where StatusCode == "Error";
let errorTraceSpans = spans | where TraceId in ((errorSpans | distinct TraceId));
// Each Error span, with the attempt it belongs to, under the span itself and its three nearest ancestors.
// The request span is one of them.
let spanErrors = errorSpans
    | project TraceId, S0 = SpanId, S1 = ParentSpanId, ErrorAttempt = Attempt
    | lookup kind=leftouter (errorTraceSpans | project TraceId, S1 = SpanId, S2 = ParentSpanId) on TraceId, S1
    | lookup kind=leftouter (errorTraceSpans | project TraceId, S2 = SpanId, S3 = ParentSpanId) on TraceId, S2
    | mv-expand SpanId = pack_array(S0, S1, S2, S3) to typeof(string)
    | where isnotempty(SpanId)
    // An Error span without an attempt, such as inbound or response transfer, fails the whole request.
    | summarize ErrorAttempts = make_set(coalesce(ErrorAttempt, 0)) by TraceId, SpanId;
OTelLogs
| where TimeGenerated > ago(lookback)
// Request logs of model calls. Tool calls have no gen_ai.request.model.
| where isnotempty(Attributes["gen_ai.request.model"])
| extend Attempt = coalesce(toint(Attributes["azure.ai_gateway.routing.attempt"]), 1),
         Status = toint(Attributes["http.response.status_code"]),
         InputTokens = coalesce(tolong(Attributes["azure.ai_gateway.client.usage.prompt_tokens"]),
                                tolong(Attributes["azure.ai_gateway.client.usage.input_tokens"])),
         CachedInputTokens = coalesce(tolong(Attributes["azure.ai_gateway.client.usage.prompt_tokens_details.cached_tokens"]),
                                      tolong(Attributes["azure.ai_gateway.client.usage.input_tokens_details.cached_tokens"])),
         OutputTokens = coalesce(tolong(Attributes["azure.ai_gateway.client.usage.completion_tokens"]),
                                 tolong(Attributes["azure.ai_gateway.client.usage.output_tokens"])),
         // The model that answered, such as a versioned name. A call that failed before an answer has only the alias.
         Model = coalesce(tostring(Attributes["gen_ai.response.model"]), tostring(Attributes["gen_ai.request.model"]))
| extend Tenant = tostring(Attributes["azure.ai_gateway.identity.claims.tid"]),
         ObjectId = tostring(Attributes["azure.ai_gateway.identity.claims.oid"]),
         // v2 tokens carry azp, v1 tokens carry appid. Both identify the client application.
         ClientApp = coalesce(tostring(Attributes["azure.ai_gateway.identity.claims.azp"]),
                              tostring(Attributes["azure.ai_gateway.identity.claims.appid"])),
         // Empty unless you add a custom claim named cost_center and your tokens carry it.
         CostCenter = tostring(Attributes["azure.ai_gateway.identity.claims.cost_center"]),
         ApiKeyId = tostring(Attributes["azure.ai_gateway.api_key_id"]),
         Alias = tostring(Attributes["gen_ai.request.model"])
// One row per request: the status and model of its last attempt, and the tokens and cost of all its attempts
| summarize arg_max(Attempt, Status, Model),
            HasUsage = countif(isnotnull(InputTokens) or isnotnull(OutputTokens)) > 0,
            InputTokens = sum(coalesce(InputTokens, long(0))),
            CachedInputTokens = sum(coalesce(CachedInputTokens, long(0))),
            OutputTokens = sum(coalesce(OutputTokens, long(0))),
            GatewayCost = sum(coalesce(todouble(Attributes["azure.ai_gateway.client.token.cost"]), 0.0)),
            Currency = take_anyif(tostring(Attributes["azure.ai_gateway.currency"]),
                                  isnotempty(Attributes["azure.ai_gateway.currency"]))
    by TraceId, SpanId, Tenant, ObjectId, ClientApp, CostCenter, ApiKeyId, Alias
| lookup kind=leftouter spanErrors on TraceId, SpanId
| extend Failed = coalesce(Status, 0) >= 400 or set_has_element(ErrorAttempts, 0) or set_has_element(ErrorAttempts, Attempt)
| summarize Requests = count(),
            FailedRequests = countif(Failed),
            RequestsWithoutUsage = countif(not(HasUsage)),
            InputTokens = sum(InputTokens),
            CachedInputTokens = sum(CachedInputTokens),
            OutputTokens = sum(OutputTokens),
            GatewayCost = round(sum(GatewayCost), 6),
            Currency = take_anyif(Currency, isnotempty(Currency))
    by Tenant, ObjectId, ClientApp, CostCenter, ApiKeyId, Alias, Model
| extend TotalTokens = InputTokens + OutputTokens
| order by TotalTokens desc

On GitHub, open infra/chargeback.kql.

1

Prepare a price list

Copy infra/prices.example.json to prices.json. Replace <model-name> with each value in the report's Model column, which is the model that answered (for example a versioned name), and copy current prices per million tokens from your provider's pricing page or agreement. A versioned name such as my-model-2026-01-01 matches the key my-model.

Caution

The example prices are placeholders. Don't publish a report until you've replaced them with real prices.

Choose your track
1

Run the query

In the Azure portal, open the Log Analytics workspace behind your Application Insights resource and select Logs. Switch from Simple mode to KQL mode, paste the contents of infra/chargeback.kql, and select Run. You get one row per tenant, object ID, client app, API key, alias, and model.

2

Export it

Select ShareExport to CSV (all columns). The browser downloads query_data.csv: move it to the lab folder and rename it usage.csv.

3

Price it

# Price the exported usage with your price list, group it by tenant, client app, and API key, and save the report
echo "๐Ÿ“Š Chargeback by tenant, client app, and API key:"
python3 infra/chargeback_report.py usage.csv prices.json --by Tenant,ClientApp,ApiKeyId --out chargeback.csv &&
  echo "๐Ÿ“„ Report saved to chargeback.csv"

On GitHub, open infra/chargeback_report.py. It sums requests, failed requests, and tokens per group, prices uncached input, cached input, and output tokens separately, and lists any model without a price. Group by any column from the query, for example --by CostCenter or --by ApiKeyId,Model.

Expected output: one row for your API key ((none) in Tenant and ClientApp) and one for your Entra identity (your tenant ID and the Azure CLI's client ID), followed by a total. For five calls per caller, the report looks like this, with your own token counts and costs:

Text
Tenant     ClientApp  ApiKeyId        Requests  FailedRequests  InputTokens  CachedInputTokens  OutputTokens  UnpricedTokens  EstimatedCost (USD)  GatewayCost (USD)
(none)     (none)     app-first-call  5         0               85           0                  797           0               0.003273             0.001615
<tenant>   <app-id>   (none)          5         0               85           0                  706           0               0.002909             0.001433

Total: 1,673 tokens, estimated 0.006182 USD

Two columns come from the gateway rather than from your price list:

  • FailedRequests counts calls that answered HTTP 4xx or 5xx, and calls with a span of their own whose status is Error. A model stream or an MCP call can fail after HTTP 200; see Operation failures. Decide whether you charge for them: a failed stream can still have consumed tokens. A call that the gateway retried or failed over counts once, by its last attempt, so a failover that answered isn't a failure. Calls that share a client trace, through a W3C traceparent header, are still judged one by one.
  • GatewayCost sums azure.ai_gateway.client.token.cost, the gateway's estimate from public prices. The logs of streamed responses carry this cost but no token usage, so they add nothing to EstimatedCost. If you sent streamed or failing traffic, for example with lab 13's traffic script, the API key row counts those calls too, and the report ends with a line such as 14 request(s) reported no token usage, such as streamed responses or failed calls.
Tip

The report shows IDs. Resolve a client app's name with az ad sp show --id <client-app-id> --query displayName -o tsv, and an API key's name under AccessAPI keys.

Checkpoint

chargeback.csv has a priced row for each caller, and no model is listed under "No price for".

Query token and cost metrics with PromQL

Token and cost metrics land in the Azure Monitor workspace (managed Prometheus) linked to Application Insights, as azure.ai_gateway.client.token.usage and azure.ai_gateway.client.token.cost. They're the same metrics the dashboard reads, and they carry the claims you enabled for Metrics as labels.

1

Find the Prometheus endpoint

# Find the Azure Monitor workspace (managed Prometheus) linked to Application Insights, and its query endpoint
AMW_ID="$(az resource show --ids "$APPI_ID" --api-version 2020-02-02 --query properties.AzureMonitorWorkspaceResourceId -o tsv)"
PROM_URL="$(az resource show --ids "$AMW_ID" --api-version 2023-04-03 --query properties.metrics.prometheusQueryEndpoint -o tsv)"
echo "๐ŸŒ Prometheus endpoint: $PROM_URL"

# Get a Microsoft Entra token for the Prometheus query API
PROM_TOKEN="$(az account get-access-token --resource https://prometheus.monitor.azure.com --query accessToken -o tsv)"
echo "๐Ÿ‘ค Token acquired: in PROM_TOKEN (${#PROM_TOKEN} characters, not shown)"
2

Query token usage for the last day

The metric and label names contain dots, so the query quotes them. The microsoft.appresourceid label limits it to your Application Insights resource.

Each gateway instance exports its own cumulative counters, labeled with service.instance.id, so the query takes the highest value of each series in the last day and adds up the instances. increase() doesn't fit lab traffic: it measures growth between samples, and a new series often appears with its first burst already counted, so a short burst reads as 0.

# Select the token usage metric of your Application Insights resource, with quoted names because they contain dots
APPI_LABEL="$(echo "$APPI_ID" | tr '[:upper:]' '[:lower:]')"
SELECTOR="{__name__=\"azure.ai_gateway.client.token.usage\",\"microsoft.appresourceid\"=\"$APPI_LABEL\"}"

# Group by model, token type, API key, tenant, and client app
BY='"gen_ai.response.model","gen_ai.token.type","azure.ai_gateway.api_key_id"'
BY+=',"azure.ai_gateway.identity.claims.tid","azure.ai_gateway.identity.claims.appid"'
BY+=',"azure.ai_gateway.identity.claims.azp"'

# Query the tokens of the last day, and print one row per group. v1 tokens carry appid, v2 tokens carry azp
echo "๐Ÿ“Š Tokens in the last day, by model, token type, API key, tenant, and client app:"
curl -sS -G "${PROM_URL%/}/api/v1/query" -H "Authorization: Bearer $PROM_TOKEN" \
  --data-urlencode "query=sum by ($BY) (max_over_time(${SELECTOR}[1d])) > 0" \
  | jq -r '.data.result[] | [.metric["gen_ai.response.model"], .metric["gen_ai.token.type"],
      .metric["azure.ai_gateway.api_key_id"], .metric["azure.ai_gateway.identity.claims.tid"],
      ((.metric["azure.ai_gateway.identity.claims.appid"] | select(. != "n/a"))
        // .metric["azure.ai_gateway.identity.claims.azp"] // "-"),
      (.value[1] | tonumber | round)] | @tsv'

Expected output: rows per model, token type, and caller, in any order. A claim that a caller doesn't have shows n/a, so your API key rows show n/a for the tenant and client app, and your Entra rows show unknown for the API key. A - for the client app marks series from before you exported appid and azp to metrics, such as lab 13's traffic.

Text
gpt-5-mini-2025-08-07  prompt_tokens                               app-first-call  n/a       n/a       85
gpt-5-mini-2025-08-07  completion_tokens                           app-first-call  n/a       n/a       809
gpt-5-mini-2025-08-07  completion_tokens_details.reasoning_tokens  app-first-call  n/a       n/a       512
gpt-5-mini-2025-08-07  total_tokens                                app-first-call  n/a       n/a       894
gpt-5-mini-2025-08-07  prompt_tokens                               unknown         <tenant>  <app-id>  85
gpt-5-mini-2025-08-07  completion_tokens                           unknown         <tenant>  <app-id>  958
gpt-5-mini-2025-08-07  total_tokens                                unknown         <tenant>  <app-id>  1043
gpt-5-mini-2025-08-07  total_tokens                                app-first-call  n/a       -         2202
Warning

Don't add up every token type. A model reports totals and details (for example total_tokens and prompt_tokens_details.cached_tokens) next to input and output, so a plain sum double counts. Use the total type if there is one, otherwise input plus output. Cached tokens are usually a subset of input tokens.

The names follow the API of each call: Chat Completions reports prompt_tokens and completion_tokens, while Responses reports input_tokens and output_tokens. gen_ai.response.model is the model name that the provider returned, so one deployment can show up as gpt-5-mini-2025-08-07 for Chat Completions and gpt-5-mini for Responses. Streamed calls report no token usage, so they have no series here, but they have a cost.

3

Query estimated cost

azure.ai_gateway.client.token.cost has one series per token type and one that covers the whole call, with an empty gen_ai.token.type. Select that one, so you don't add up the same cost twice.

# Select the cost series that cover a whole call: their gen_ai.token.type label is empty
COST_SELECTOR="{__name__=\"azure.ai_gateway.client.token.cost\",\"gen_ai.token.type\"=\"\","
COST_SELECTOR+="\"microsoft.appresourceid\"=\"$APPI_LABEL\"}"

# Group by model, API key, tenant, client app, currency, and whether a price was missing
COST_BY='"gen_ai.response.model","azure.ai_gateway.api_key_id","azure.ai_gateway.identity.claims.tid"'
COST_BY+=',"azure.ai_gateway.identity.claims.appid","azure.ai_gateway.identity.claims.azp"'
COST_BY+=',"azure.ai_gateway.currency","azure.ai_gateway.price_missing"'

# Query the cost of the last day, and print one row per group
echo "๐Ÿ“Š Estimated cost in the last day, by model, API key, tenant, client app, and price status:"
curl -sS -G "${PROM_URL%/}/api/v1/query" -H "Authorization: Bearer $PROM_TOKEN" \
  --data-urlencode "query=sum by ($COST_BY) (max_over_time(${COST_SELECTOR}[1d])) > 0" \
  | jq -r '.data.result[] | [.metric["gen_ai.response.model"], .metric["azure.ai_gateway.api_key_id"],
      .metric["azure.ai_gateway.identity.claims.tid"],
      ((.metric["azure.ai_gateway.identity.claims.appid"] | select(. != "n/a"))
        // .metric["azure.ai_gateway.identity.claims.azp"] // "-"),
      .metric["azure.ai_gateway.price_missing"], .metric["azure.ai_gateway.currency"],
      (.value[1] | tonumber * 1000000 | round / 1000000)] | @tsv'

Expected output: one row per model and caller, in the currency usd. Add up a caller's rows: the total matches its GatewayCost in the chargeback report. Lab 13's traffic adds API key rows for gpt-5-mini, from streamed Responses calls that have a cost but no token series, and for unknown, from streams that failed before a model answered.

Text
gpt-5-mini-2025-08-07  app-first-call  n/a       n/a       true   usd  0.001639
gpt-5-mini-2025-08-07  unknown         <tenant>  <app-id>  true   usd  0.001937
gpt-5-mini-2025-08-07  app-first-call  n/a       -         true   usd  0.004248
gpt-5-mini             app-first-call  n/a       -         false  usd  0.002518
unknown                app-first-call  n/a       -         false  usd  0.000338

azure.ai_gateway.price_missing is true when a token type of the call has no public price, so the cost is a lower bound. Chat Completions calls can show true even when every token you used has a price: the call also reports categories such as completion_tokens_details.accepted_prediction_tokens, which have no price, even at 0 tokens.

Note

The query above is exact for series that started in the last day, and also counts the earlier traffic of an instance that ran longer. The docs show PromQL in its underscore form, sum by (gen_ai_request_model) (gen_ai_client_token_usage), which is how some OTLP backends store these names. If a query returns nothing, list the names your backend stored with count by (__name__) ({__name__=~".*token.*"}).

Verify

1

Compare the report with the dashboard

Open MonitoringModels with 24H selected. The cost, tokens, and calls of your two callers in Model usage by caller identity are in the same range as their GatewayCost and token columns in chargeback.csv.

2

Confirm the client app label

Run the PromQL queries again. Your Entra rows show a client app ID, which comes from the Metrics signal you enabled for appid and azp, and each caller's cost adds up to its GatewayCost.

Go further

  • Enforce what you measure. A cost limit can give a different budget to a specific Entra identity or to a Custom token claim combination, such as an appid value. See Enforce cost budgets with per-caller overrides.
  • Send telemetry to another backend. In the setup wizard, Custom OpenTelemetry destination takes a Metrics endpoint, Logs endpoint, and Traces endpoint, such as a collector, Grafana, Datadog, or any OTLP service. Authenticate with Custom headers, whose saved values the API doesn't return, or a Managed identity with the receiver's token audience. Claims are exported the same way. The built-in dashboard only reads Application Insights, so with a custom destination you build these views in that platform. See Send telemetry to a custom destination and Attribute estimated model costs.
  • Fan out with a collector. To keep claims in one place and send them to several backends, point the gateway at your own OpenTelemetry Collector and route from there.
  • Keep metric cardinality in check. Every distinct claim value is a new series. Put high-cardinality claims such as oid on logs and traces for detail, and keep metrics to tid, client apps, and bounded custom claims.
  • Schedule the report. Run the REST track monthly from a pipeline with a managed identity that has Log Analytics Reader, and store chargeback.csv with your finance data.

Clean up

The exported files contain tenant and object IDs. Treat them as personal data.

Choose your track
1

Restore the preset

In MonitoringSettingsLoggingToken claim attributes, clear Include appid in metrics and Include azp in metrics. Saving changesโ€ฆ shows while the portal saves each change. Remove any custom claim you added with its row's More actions menu โ€บ Remove. Close and reopen Settings to confirm.

2

Delete the exported files

Delete usage.csv, prices.json, and chargeback.csv from the lab folder.

Keep monitoring on if you continue: the agent labs use it to show where agent calls went. Otherwise, follow the clean-up steps in Monitor usage, cost, and traces. azd down and terraform destroy don't touch the gateway and monitoring from the earlier labs, because the Bicep + azd and Terraform tracks created their own.

Next steps

Finished the lab?

Mark it complete to track your progress across paths. It's saved only in this browser.