Observability

The runtime emits OpenTelemetry traces and metrics so you can see what an agent and its tools actually did — and tell whether a failure is the app's fault or the platform's — without writing any telemetry code. This page explains how to turn it on, what we emit, and how the attribute names are structured.

App authors don't need to read this to get value: with telemetry enabled, the standard Application Insights views light up automatically. The optional dashboards/workbooks that make monitoring even easier are in Appendix A.

Turn it on (no code)

The app's function_app.py stays two lines — you enable telemetry through configuration only.

Minimum (path of least resistance):

1. APPLICATIONINSIGHTS_CONNECTION_STRING — already present on most function apps.
2. Install the worker exporter extra: pip install azurefunctions-agents-runtime[monitor]. The runtime configures Azure Monitor automatically when that extra is installed and no OpenTelemetry provider is already active.

That's it — when a real OpenTelemetry provider is active, the runtime's invocation and dynamic_session.execute spans and metrics flow to Application Insights. Without the [monitor] extra, the runtime's worker spans are not exported unless another OpenTelemetry provider is already active, and if a connection string is set in that state the runtime logs a warning telling you to install azurefunctions-agents-runtime[monitor].

Optional — unified host + worker correlation:

Add "telemetryMode": "OpenTelemetry" to host.json if you also want the Functions host to export in OpenTelemetry format, so host request telemetry and the runtime's worker spans share one end-to-end operation. This is additive — it is not required for the agent/tool spans themselves. Avoid turning on both a worker exporter and host worker-streaming (PYTHON_APPLICATIONINSIGHTS_ENABLE_TELEMETRY) for the same data, which can double telemetry volume. The runtime now auto-detects an already-configured OpenTelemetry provider and skips its own Azure Monitor setup, so the worker path will not double-export — though it is still unnecessary when the runtime is already configuring the exporter.

create_function_app() calls _observability.configure_observability() to configure the optional Azure Monitor exporter when available and no OpenTelemetry provider is already active. This shared bootstrap does not import or initialize an agent harness. Runtime span/content helpers continue using the active OpenTelemetry provider or no-op when none is active; any selected SDK-native instrumentation is initialized by the selected harness only.

Where the runtime's output shows up (traces, not logs)

The runtime is instrumented span-first, so its telemetry surfaces as spans — look in Transaction Search / the end-to-end transaction view, not the "Logs" (AppTraces) list.

The runtime's own log lines are intentionally not in your Application Insights traces: its internal logger sits under the azure.functions.* namespace, which the Functions Python worker classifies as system logs (emitted with customer_app_insight = false), so they never reach your app's traces. host.json logLevel does not surface them, and worker-side DEBUG would require the PYTHON_ENABLE_DEBUG_LOGGING app setting. This is by design.

So to debug a run, use the spans — not the log list:

• The runtime-owned invocation span, dynamic_session.execute, and tool spans, with the af.* attributes below.
• Runtime span events on the invocation span (input/response-contract milestones — see that span's section) and any framework- or SDK-owned child spans.
• Failures are captured on the span, not just in logs: a failing run is marked error with record_exception + af.fault_domain, so it is fully visible in the transaction view and the AppExceptions table.

The host lifecycle lines you do see in the logs list (Executing…, Executed… (Succeeded), Function duration…) come from the Functions host — a separate telemetry plane from the runtime's worker spans. Use the Quick KQL below to pull a whole run together.

Attribute naming: what af. means

Custom runtime attributes use the af. prefix — short for Azure Functions agents. Standard OpenTelemetry and GenAI attributes are reused when their semantics match the value. This distinction:

• keeps custom runtime attributes separate from standard names,
• lets queries use standard names for reported model, token-usage, agent, and tool metadata.

Four sub-namespaces group the detail, plus two cross-cutting attributes:

Namespace	Used for
af.agent.*	attributes on the runtime-owned invocation span
af.dynamic_session.*	attributes on the dynamic_session.execute (code sandbox) span
af.web_request.*	attributes on the web_request (outbound HTTP tool) span
af.delegate.*	attributes on the execute_tool delegate_<slug> span (chat-time sub-agent delegation)
af.fault_domain, af.lifecycle_stage	cross-cutting; may appear on runtime-owned spans

Where a standard OpenTelemetry attribute already exists we reuse it instead of inventing an af. name — for example server.address for the session-pool host. Standard GenAI attributes are emitted only where the runtime has matching evidence for provider, model, agent, conversation, tool, or usage semantics. The runtime does not fabricate model spans or timing when the selected SDK does not report a reliable model boundary.

Spans and attributes we emit today

Cross-cutting af.* (any runtime span)

Attribute	Meaning
af.fault_domain	Whose fault a failure is: app, runtime, platform, model, connector, sandbox, web_request, delegate, unknown. Set only on failing spans.
af.lifecycle_stage	Which run stage the span represents, e.g. agent_run, tool_execution.

Runtime-owned invocation span

One per agent invocation (timer, connector, HTTP, …). This is the runtime-level run span for af.agent.* attributes, lifecycle events, and aggregate tool/error counts. Use these attributes, not the span name alone, for dashboards that need to work across runtime versions and selected harnesses. It contains harness execution and tool spans; Copilot tools are nested under the semantic execution span described below.

Attribute	Meaning
af.agent.name	Canonical agent slug, not an app-qualified display label or frontmatter display name.
af.agent.trigger_type	timer, connectorTrigger, http, …
af.agent.model	Model/deployment used for the run, when emitted on the runtime invocation surface.
af.agent.session_id	Conversation/session id, when emitted on the runtime invocation surface.
af.agent.harness	Selected app-level harness for this runtime-owned invocation span: maf or copilot. This is an attribute, not a synthetic span.
af.agent.outcome	success or error.
af.agent.tool_call_count	Number of tool calls in the run.
af.agent.tool_error_count	Tool calls that failed — includes a "successful" call whose result carried an error or non-empty stderr, plus any delegate_<slug> call the coordinator recovered from (a specialist failure or timeout — see Span execute_tool delegate_<slug>).
af.agent.input_bytes	Size of the trigger payload / HTTP body.
af.agent.response_bytes	UTF-8 byte length of the model's final response.
af.agent.input	The trigger payload / HTTP body. Content — only when ENABLE_SENSITIVE_DATA=true.
af.agent.response	The final response text. Content — only when ENABLE_SENSITIVE_DATA=true.

Plus af.lifecycle_stage=agent_run, and af.fault_domain if the run fails.

For SDK-reported tool and usage events, the runtime records only public, content-free metadata. Timed tool spans require coherent start and completion records; incomplete or ambiguous timing is not guessed. Each unique usage event is attached to the owning runtime invocation span (or Copilot semantic span when no runtime span exists) as af.agent.model.usage with standard gen_ai.usage.* fields where semantics match. Counting a tool once in telemetry is not a guarantee of exactly-once tool effects.

Outer runtime invocation spans are intentionally harness-neutral. They use agent.run {slug} and the af.agent.* attributes regardless of the selected backend. MAF's framework-owned GenAI instrumentation is unchanged. Copilot supplies one semantic execution child span, not a duplicate outer summary span, as described below. Neither path fabricates unavailable provider/model hierarchy.

Copilot semantic execution span

This follows FRD 0009 §4.10.

Each direct, streaming, delegate, or workflow-leaf Copilot execution emits one INTERNAL invoke_agent {readable-agent-name} span beneath agent.run {slug} when that outer span exists. Its duration covers adapter setup through cleanup, not model inference. Direct runner calls create only the semantic span beneath their available caller context.

Attribute	Meaning
gen_ai.operation.name	invoke_agent.
gen_ai.agent.id	Existing stable agent ID, as defined below.
gen_ai.agent.name	Existing app-qualified readable name, as defined below.
gen_ai.conversation.id	Resolved public session ID, never the native session/storage identifier.
gen_ai.request.model	Configured model, only when known.

Custom and event-derived tool spans use the semantic invocation context. Runtime summaries, lifecycle events, usage, and skill observations stay on the outer runtime span explicitly bound by the invocation boundary; otherwise they attach to the semantic span. Matching public names and attributes do not establish ownership. Invocation token totals are optional and currently deferred.

The span uses the existing telemetry pipeline and privacy/error contracts. Unavailable provider data, model-call spans, inference latency, metrics, cost, and native topology are not inferred. This change adds no file bridge, receiver, exporter, telemetry setting, or native telemetry opt-in.

Stable agent ID

When the runtime emits gen_ai.agent.id, it uses a deterministic, human-readable <site-name>/<agent-slug> string. The site name comes only from trimmed, lower-cased WEBSITE_SITE_NAME; an unset, empty, or whitespace-only value uses local. The agent name is the canonical filename-derived runtime slug, not the authored frontmatter display name:.

For example, WEBSITE_SITE_NAME= Contoso-Agents and agent file agents/billing.agent.md produce contoso-agents/billing. Locally, the same agent is local/billing. Owner, deployment, resource-group, and resource-ID values do not affect the ID.

Identical site-name and slug inputs produce the same ID for fresh agents. Renaming the site changes the ID. Local projects with the same slug share an ID; this is not a globally unique Azure resource identifier.

The same value is emitted in the agent_runtime_indexed startup summary as agent_id so operators can copy it when registering agents with external services such as A2A, Agent 365, or Foundry.

Readable agent name

When the runtime emits gen_ai.agent.name, it keeps it separate from the stable gen_ai.agent.id. For primary agents, streaming, chat delegates, and workflow leaf specialists, the name is <Function App name>/<canonical agent slug> when the trimmed WEBSITE_SITE_NAME is non-blank. Site-name casing is preserved; owner, deployment id, and the frontmatter display name: do not choose this label. For example, WEBSITE_SITE_NAME=Contoso-Agents gives the billing agent the name Contoso-Agents/billing, while its ID is contoso-agents/billing. Some harnesses can also use this readable name as response author metadata; it is not a new canonical runtime identity.

An unset, empty, or whitespace-only site name preserves the caller's canonical slug, or None when the caller omitted the name. With a site name, an omitted or empty caller name uses the same main suffix as the ID. Qualification does not change files, routes, delegate_<slug> tool names, Catalog lookup, history/session scope, locks, workflow authorization, or backend storage namespaces. Runtime af.agent.name stays the canonical slug.

Span events (runtime lifecycle milestones)

These runtime-owned invocation-span events mark input/output-contract boundaries. They carry only non-sensitive metadata (names/status/counts — never request/response/model content). Harness-owned child spans and usage events carry per-model/per-tool detail when the selected SDK reports it.

• af.input.validation_failed — HTTP input-schema validation failed before agent execution; includes af.fault_domain=app and af.http.status_code.
• af.response.invalid_json — the agent completed, but its HTTP response could not be parsed as the required JSON contract; includes af.fault_domain=app.
• af.response.schema_validation_failed — the agent completed, but the parsed HTTP JSON response failed schema validation; includes af.fault_domain=app.
• af.agent.invoke.completed — _run_agent(...) returned successfully and the runtime is handling the final response contract.

Runtime source coverage

These expectations apply to runtime-captured turn telemetry:

Source	Expectation
Direct HTTP and built-in chat UI turns	One runtime-owned invocation span per completed turn, with tool accounting and usage observations attached from public events where available.
/chatstream SSE	The stream reflects callbacks for the same turn; it does not imply full native child-session hierarchy or session-open telemetry.
delegate_<slug> calls	The host records the delegate tool boundary and child invocation telemetry where available. Coordinator SSE still exposes only the delegate tool boundary and final child text.
Workflow Sub Agent leaves	Each Copilot leaf has its own semantic execution span, including adapter setup/cleanup; its configured model is attached when known. No inference span or native model timing is fabricated.
Non-HTTP triggers, inbound MCP, and Debug UI continuation	When they route through the shared runner, they follow the same turn-scoped accounting rules.

Span dynamic_session.execute

One per fhs_execute_python call, as a child of the runtime-owned invocation span.

Attribute	Meaning
server.address	Session-pool host (OTel semconv).
af.operation_id	Correlation id also sent to ACA in the operation-id header.
af.dynamic_session.session_id	ACA dynamic-session id.
af.dynamic_session.code_bytes	Size of the submitted code.
af.dynamic_session.stdout_bytes	Size of stdout.
af.dynamic_session.stderr_bytes	Size of stderr.
af.dynamic_session.stderr_present	true ⇒ the code failed (this is what used to be invisible).
af.dynamic_session.session_reused	Whether the ACA session already existed.
af.dynamic_session.setup_ran	Whether the one-time session setup ran this call.
af.dynamic_session.code / .stdout / .stderr	Content — only when ENABLE_SENSITIVE_DATA=true.

Plus af.lifecycle_stage=tool_execution. When stderr is present or the call throws, the span is marked ERROR with af.fault_domain=sandbox. This is the key fix: a broken execution no longer looks like a successful tool call.

Span web_request

One per web_request tool call, as a child of the runtime-owned invocation span. Attributes are deliberately host-only — the full URL (with query string, and any userinfo) is never attached to the span, and secrets are never logged, regardless of ENABLE_SENSITIVE_DATA.

Attribute	Meaning
http.request.method	HTTP verb used (GET, POST, …).
server.address	Target host (OTel semconv) — set once the SSRF validator has approved a host.
url.scheme	http or https.
http.response.status_code	Response status code, when a response was received.
af.web_request.blocked_reason	Present only when the SSRF validator rejects the request (e.g. private_ip, imds, allowlist_denied).
af.web_request.response_bytes	Size of the response body actually read (before truncation applies).
af.web_request.body_truncated	true when the response exceeded max_response_bytes and was truncated.

Plus af.lifecycle_stage=tool_execution. SSRF rejections, timeouts, and transport errors all mark the span ERROR with af.fault_domain=web_request.

Span execute_tool delegate_<slug> (chat-time sub-agent delegation)

Chat-time delegation (FRD 0007) records the coordinator's delegate_<slug> tool boundary and annotates it with af.delegate.* attributes. When the selected harness supplies a nested child invocation, that child span is parented by the active OpenTelemetry context. Copilot captures the semantic invocation context explicitly for tool callbacks, preserving the delegate tool -> specialist execution hierarchy.

Attribute	Meaning
af.delegate.specialist	The specialist's slug (the same identity used for its delegate_<slug> tool name).
af.delegate.outcome	success, error, timeout, or cancelled.
af.delegate.task_bytes	Size of the task argument passed to the specialist.
af.delegate.response_bytes	Size of the specialist's response text (only set on success).
af.delegate.task / .result	Content — only when ENABLE_SENSITIVE_DATA=true.

Plus af.fault_domain=delegate on a failing span: the specialist raised (including a failure constructing the specialist itself), or the effective delegation timeout — min(specialist timeout, coordinator's remaining time) — was exceeded. A parent/request cancellation (asyncio.CancelledError) is different: the handler tags the span outcome=cancelled and still counts it in the delegate call metric (it was genuinely dispatched), but never converts it into a recoverable error — it re-raises immediately and aborts the whole run, rather than being recorded as a delegate error (FRD 0007 Decision #12).

Error accounting. _looks_like_tool_error (the sandbox/web_request JSON {"error": …} / non-empty-stderr heuristic) does not understand a specialist's sanitized free-text failure message, so relying on it alone would silently under-count. The delegated adapter tracks its own recoverable failures explicitly and folds them into af.agent.tool_error_count on top of the heuristic's count.

Accepted limitations (v1): - SSE is a black box at the boundary. The coordinator's stream emits tool_start/tool_end for delegate_<slug> (task in, final text out) exactly like the sandbox/web_request tools; a specialist's own internal deltas and nested tool calls do not surface on the wire. - Token usage does not roll up across the boundary. The delegate tool span carries no usage, and a specialist's tokens are not merged into the coordinator's totals. Sum child spans or usage events in the backend for a combined per-request total.

Metrics

Namespace azure_functions_agents.*:

Metric	Meaning
azure_functions_agents.dynamic_session.executions	Count of fhs_execute_python calls.
azure_functions_agents.dynamic_session.errors	Count that failed or produced stderr.
azure_functions_agents.web_request.requests	Count of web_request tool calls.
azure_functions_agents.web_request.errors	Count that were blocked by the SSRF validator, timed out, or otherwise failed.
azure_functions_agents.delegate.calls	Count of delegate_<slug> tool invocations (chat-time sub-agent delegation).
azure_functions_agents.delegate.errors	Count that failed, raised, or timed out (specialist-side; sanitized before reaching the model).

Sensitive data

Sensitive-data capture is controlled by the single ENABLE_SENSITIVE_DATA environment variable, default off.

• Off (default): only metadata is recorded — sizes, counts, outcome, fault domain. The *_bytes attributes above are emitted; the content attributes are not.
• On: content attributes are attached (bounded in length): af.agent.input, af.agent.response, af.dynamic_session.code / .stdout / .stderr, and af.delegate.task / .result, plus harness-owned prompt/response/tool-argument content when that harness supports the shared setting. Runtime run-log payloads add response content only when this switch is on. Raw tool-call records follow the selected harness policy: MAF can include them when the switch is on, while Copilot run logs keep them excluded.
• Never captured, regardless of the flag: secrets — MCP Authorization headers/tokens, connection strings, raw native envelopes, reasoning text, skill paths or content, and the ACA system key. Endpoints are reduced to host only. The web_request span never carries the full request URL (query string or userinfo stripped), request/response bodies, or header values — it is host/status/size metadata only, unaffected by ENABLE_SENSITIVE_DATA.
• Exception diagnostics: recorded Python exceptions may include normal OpenTelemetry exception.message / exception.stacktrace and af.fault_domain even when ENABLE_SENSITIVE_DATA is off. This does not widen prompt/request/response/tool-argument capture, but arbitrary exception text can still contain sensitive data if application code includes it. Unexpected Copilot SDK startup, session-create/resume, and turn failures retain their original Python exception, traceback, and cause, matching MAF. Runtime exception logs include the operation name, without separately dumping requests or SDK payloads. Standard HTTP/SSE/MCP error handling also uses the original exception text, so SDK exceptions containing sensitive data can expose it in responses as well as logs and telemetry. ENABLE_SENSITIVE_DATA does not suppress exception diagnostics. Native SDK logging/export remains disabled.

Best-effort diagnostics and telemetry cleanup are intentionally non-throwing: they must not change request results, cancellations, or SSE termination.

HostedSkill usage logs

The azure.functions.AgentRuntime logger emits two content-free INFO records for HostedSkill usage, independently of OpenTelemetry span export:

• HostedSkill binding defined is emitted when @app.hosted_skill successfully decorates a handler during app indexing. It includes execution_surface=hosted_skill_binding, agent_name, handler, and arg_name.
• HostedSkill binding executed is emitted when validated run() or consumed stream() execution begins. It includes execution_surface=hosted_skill_binding, agent_name, and mode (run or stream). Failed backend attempts still retain this execution record.

These records never include prompts, responses, session IDs, model configuration, or tool data, regardless of ENABLE_SENSITIVE_DATA.

Noise & cost control

Most Application Insights volume from a function app is not the runtime's spans — on a real app, runtime and harness spans were only a few KB/run, while over 90% of ingestion was low-signal traces: Azure SDK HTTP request/response dumps, the Azure Monitor exporter's own "Transmission succeeded…" logs, credential chatter, and Functions host startup dumps.

Worker-side noise — handled for you. Regardless of whether telemetry export is active, the runtime raises the log level of known-noisy third-party loggers (Azure SDK HTTP logging, the Azure Monitor exporter, azure.identity, httpx, and OpenTelemetry internals) — but only when no level is set directly on that logger, so a level you set directly on it is never overridden (a level set on a parent/root logger is not consulted). See _NOISY_LOGGERS in _observability.py.

Host-side noise — set it in host.json. Host startup/options logging is emitted by the Functions host, so quiet it with log-level overrides (or the equivalent AzureFunctionsJobHost__logging__logLevel__<category> app settings):

{
  "logging": {
    "logLevel": {
      "default": "Warning",
      "Host.Startup": "Warning",
      "Host.Function.Console": "Warning",
      "Microsoft.Azure.WebJobs.Hosting.OptionsLoggingService": "Warning"
    }
  }
}

Other levers: keep ENABLE_SENSITIVE_DATA off (default); sample with OTEL_TRACES_SAMPLER=parentbased_traceidratio + OTEL_TRACES_SAMPLER_ARG=0.1; set a daily cap / retention on the workspace; and avoid double export — don't run both the worker exporter and host worker-streaming (PYTHON_APPLICATIONINSIGHTS_ENABLE_TELEMETRY + telemetryMode) for the same telemetry. The runtime now detects an existing OpenTelemetry provider and skips its own Azure Monitor setup, so enabling the worker exporter path will not double-export — it is simply unnecessary when the runtime is already handling exporter configuration.

Delegation-heavy apps: a single coordinator turn with several subagents: can fan out into many child spans (coordinator + N specialists + harness-owned model/tool children — see Span execute_tool delegate_<slug>). Azure Monitor's default rate-limited sampler counts spans, so a large fan-out can exhaust its budget and drop whole traces under load — prefer the explicit OTEL_TRACES_SAMPLER=parentbased_traceidratio + OTEL_TRACES_SAMPLER_ARG setting above instead of relying on the default. Because that sampler is trace-id-deterministic, a sampling decision applies consistently to the whole nested trace (no half-traces), but logs on a dropped trace are dropped with it.

Quick KQL

// Everything that happened in one run, in order
union AppRequests, AppDependencies, AppTraces, AppExceptions
| where OperationId == "<operation-id>"
| project TimeGenerated, itemType, Name, Message, Success, DurationMs
| order by TimeGenerated asc

// Sandbox executions that actually failed (even if the tool "succeeded")
AppDependencies
| where Name == "dynamic_session.execute"
| extend stderr_present = tostring(Properties["af.dynamic_session.stderr_present"])
| where Success == false or stderr_present == "true"
| project TimeGenerated, OperationId, DurationMs, Properties

// Delegate calls that failed or timed out (recovered — the coordinator kept running)
AppDependencies
| where Name startswith "execute_tool delegate_"
| extend outcome = tostring(Properties["af.delegate.outcome"]), specialist = tostring(Properties["af.delegate.specialist"])
| where outcome in ("error", "timeout")
| project TimeGenerated, OperationId, specialist, outcome, DurationMs, Properties

Measuring telemetry volume (billed bytes per run)

Use these to size ingestion/cost before and after enabling observability. _BilledSize is the billed bytes per item.

// Average billed volume per agent run, broken down by table.
// A "run" = an operation that contains an invocation span for this app role.
let runs = AppDependencies
| where TimeGenerated > ago(30d)
| where AppRoleName == "func-agents-6q2arnxahkobm"        // <-- your function app role name
| where Name startswith "agent.run "
| distinct OperationId;
union withsource=TableName AppRequests, AppDependencies, AppTraces, AppExceptions, AppMetrics
| where TimeGenerated > ago(30d)
| where OperationId in (runs)
| summarize items = count(), billedBytes = sum(_BilledSize) by OperationId, TableName
| summarize avgItemsPerRun = avg(items), avgBytesPerRun = avg(billedBytes) by TableName
| order by avgBytesPerRun desc

// Bucket span volume by emitted span-name family only.
// `invoke_agent` / `execute_tool` names come from SDK/framework-owned or
// runtime tool telemetry, so these labels stay descriptive rather than
// claiming invocation ownership.
let runs = AppDependencies
| where TimeGenerated > ago(7d)
| where AppRoleName == "func-agents-6q2arnxahkobm"
| where Name startswith "agent.run "
| distinct OperationId;
AppDependencies
| where TimeGenerated > ago(7d)
| where OperationId in (runs)
| summarize
    runtimeNamedItems = countif(Name == "dynamic_session.execute" or Name startswith "agent.run "),
    runtimeNamedBytes = sumif(_BilledSize, Name == "dynamic_session.execute" or Name startswith "agent.run "),
    invokeAgentNamedItems = countif(Name startswith "invoke_agent "),
    invokeAgentNamedBytes = sumif(_BilledSize, Name startswith "invoke_agent "),
    executeToolNamedItems = countif(Name startswith "execute_tool "),
    executeToolNamedBytes = sumif(_BilledSize, Name startswith "execute_tool "),
    chatNamedItems = countif(Name startswith "chat "),
    chatNamedBytes = sumif(_BilledSize, Name startswith "chat ")
  by OperationId
| summarize avgRuntimeNamedItems = avg(runtimeNamedItems), avgRuntimeNamedBytes = avg(runtimeNamedBytes),
            avgInvokeAgentNamedItems = avg(invokeAgentNamedItems), avgInvokeAgentNamedBytes = avg(invokeAgentNamedBytes),
            avgExecuteToolNamedItems = avg(executeToolNamedItems), avgExecuteToolNamedBytes = avg(executeToolNamedBytes),
            avgChatNamedItems = avg(chatNamedItems), avgChatNamedBytes = avg(chatNamedBytes)

---

Appendix A — Optional app-level enhancements

These make monitoring easier for app authors but are not required — share them if someone wants richer dashboards. They build on the spans/metrics above.

• Tier 0 — Native portal (automatic). With telemetry on, Transaction Search, Application Map, Failures, and the GenAI/Agents views work with no queries, because the runtime emits invocation spans plus harness-owned child spans where available.
• Tier 1 — One azd-provisioned Workbook. An Azure Monitor Workbook (Microsoft.Insights/workbooks) wired to the app's App Insights: runs over time, failure rate, tool failure rate, dynamic-session errors, token usage, and a failed-runs drilldown by operation_Id. A reference module ships with the sample app at infra/app/observability-workbook.bicep.
• Tier 2 — Optional alerts (off by default). Bicep metric/scheduled-query alert rules for run failure-rate and dynamic-session error-rate.
• Tier 3 — Metrics charts (no KQL). Build portal charts from the azure_functions_agents.* metrics by picking a metric and dimension.
• Tier 4 — KQL pack. Copy-paste queries (see "Quick KQL" above) for deep dives.

Appendix B — Roadmap

Implemented (must-have): runtime-owned OTel bootstrap + Azure Monitor auto-configuration via the optional [monitor] extra; fault-domain / lifecycle-stage conventions; sandbox truth-telling + ACA correlation; the agent.run summary span; sensitive-data gating (default off); chat-time sub-agent delegation span enrichment + error accounting (FRD 0007).

Follow-ups (good-to-have):

• Broader fleet metrics (per-agent run counts/duration, tokens, tool-call metrics).
• Full stage attribution for registration, model build, tool/MCP dropped-tool, storage, and response-validation failures (today's remaining silent gaps).
• Automatic per-tool span enrichment via the tool wrapper.
• The Tier-1/2 dashboards and alerts above, shipped as reusable infra.
• Delegation SSE passthrough (surface a specialist's internal stream deltas through the coordinator's stream) and cross-boundary token roll-up (FRD 0007 §4.12 accepted limitations).

Implementation map

Area	Files
Bootstrap, conventions, helpers, metrics	src/azure_functions_agents/_observability.py
Bootstrap call site	src/azure_functions_agents/app.py
Sensitive-data env handling (ENABLE_SENSITIVE_DATA)	src/azure_functions_agents/_observability.py
Sandbox span + stderr surfacing + ACA correlation	src/azure_functions_agents/system_tools/sandbox.py
web_request span, SSRF blocking, and truncation reporting	src/azure_functions_agents/system_tools/web_request.py
agent.run span + sensitive-log gating	src/azure_functions_agents/registration/_handlers.py
Copilot semantic invocation, usage/skill observations, and explicit tool parenting	src/azure_functions_agents/harness/copilot_sdk/_copilot_execution.py, _copilot_observability.py, _copilot_tool_calls.py
delegate_<slug> tool build, failure/cancellation adapter, span annotation (FRD 0007)	src/azure_functions_agents/runner.py
Tests	tests/test_observability.py, tests/test_system_tools_sandbox.py, tests/test_web_request.py, tests/test_runner_delegation.py