Skip to content

🚀 Deployment Guide

Use this page as the canonical installation guide. Start with Basic Deployment for a simple environment, or Zero Trust Deployment when network isolation is required.

Note: You can change parameter values in main.parameters.json or set them with azd env set before running azd provision. This applies only to parameters that support environment variable substitution.

Underlying infrastructure: GPT-RAG provisions the Azure AI Landing Zone (AILZ) Bicep module as its infrastructure foundation. For the full list of parameters, opt-in features (IP allow-lists, BYO Private DNS / Log Analytics, hub-and-spoke integration, etc.), and the v2 migration path, see the AILZ parameterization reference and the v2-migration guide.

Prerequisites

Required Permissions:

  • Azure subscription with Contributor and User Access Admin roles
  • Agreement to Responsible AI terms for Azure AI Services

Required Tools:

Basic Deployment

Quick setup for demos without network isolation. In this mode, the workstation can run the full flow: provision, post-provision configuration, and service deployment.

azd init -t azure/gpt-rag
az login
azd auth login
azd env set NETWORK_ISOLATION false
azd provision
azd deploy

Add --tenant for az or --tenant-id for azd if you want a specific tenant.

Resource naming: Starting with GPT-RAG v3.1.0 and AI Landing Zone v2.2.0, fresh deployments name resources using the Cloud Adoption Framework pattern (for example cosmos-<hash>-<env>-<region>-001). No extra variables are required. If you need to keep the pre-v3.1.0 names, set RESOURCE_NAMING_MODE=legacy before azd provision. See the resource naming guide for details, override options, and a before/after table.

azd provision runs GPT-RAG preflight checks before Azure Resource Manager deployment starts. These checks validate the selected region, jumpbox VM SKU restrictions, provider/location support for AI Search, Cosmos DB, Container Apps, and AI Foundry/Cognitive Services, and Azure OpenAI model quota for the configured deployments. If model quota is insufficient, the hook fails early and suggests candidate regions when possible.

Some transient Azure capacity failures are not exposed by reliable pre-create APIs. For example, Cosmos DB can still fail later with regional high-demand ServiceUnavailable; the preflight reports this limitation explicitly. Use GPT_RAG_REGIONAL_PREFLIGHT_SKIP=true only to bypass GPT-RAG regional checks, or PREFLIGHT_SKIP=true to bypass all preflight hooks.

For current published GPT-RAG umbrella releases, the postProvision hook runs locally after azd provision, and azd deploy deploys the UI, orchestrator, and ingestion services in the classic Container Apps topology.

Chat runtime modes

Exact matrix pinned; runtime validation and release remain blocked

UI v2.6.0, orchestrator v4.0.0, ingestion v2.7.0, and AILZ v2.5.0 are pinned by the umbrella integration at their exact release commits. Classic, hosted/no-panel, and explicitly selected hosted-panel are supported topologies. The manifest's umbrella tag remains unreleased; use a GPT-RAG source or release that contains these pins. Continuity, user-history, owner-binding validation, and operator-surface evidence gates remain deployment-published false. The agent version became active in the latest runtime attempt, but session readiness returned HTTP 424. The integration is not runtime-validated or shipped. See the exact integration matrix.

The platform implementation resolves one canonical topology before provisioning and materializes the corresponding legacy flags and App Configuration values. Topology never changes automatically during a chat request.

Environment or operator choice Resolved settings Resulting topology
Genuinely fresh environment DEPLOYMENT_TOPOLOGY=hosted-no-panel, DEPLOY_HOSTED_AGENT_ORCHESTRATION=true, DEPLOY_ADMINISTRATIVE_PANEL=false, CHAT_BACKEND=hosted_agent Web UI and ingestion remain in Container Apps. Chat runs in a Microsoft Foundry hosted agent. No orchestrator Container App or panel-only Cosmos DB is provisioned.
Existing environment with persisted topology Existing topology and CHAT_BACKEND stay sticky. An unmarked pre-cutover environment resolves to classic. Upgrade does not implicitly migrate identity, conversation, authorization, or cost semantics.
Explicit Container Apps fallback DEPLOYMENT_TOPOLOGY=classic, materialized hosted and panel flags false, CHAT_BACKEND=orchestrator UI routes chat to the orchestrator Container App. Classic history and panel data remain available.
Explicit migration to hosted/no-panel DEPLOYMENT_TOPOLOGY=hosted-no-panel, delegated hosted scope configured, panel false Runs the two-phase hosted lifecycle below, then validates the hosted request path before the classic chat path is removed or deactivated.
Explicit hosted panel DEPLOYMENT_TOPOLOGY=hosted-panel, hosted and administrative-panel flags true, CHAT_BACKEND=hosted_agent Deploys UI and ingestion, omits the orchestrator Container App, and provisions only the owner-index and feedback metadata containers. User-history and operator routes remain off/503 because their evidence gates stay false.

Hosted-panel is never selected implicitly. Operators must set DEPLOYMENT_TOPOLOGY=hosted-panel or explicitly set both legacy hosted and administrative-panel flags to true. A stray panel flag while hosted orchestration is false remains classic.

The deployment hooks publish the shared runtime contract under the App Configuration label gpt-rag:

Setting Operator contract
DEPLOYMENT_TOPOLOGY Canonical deployment choice: hosted-no-panel, hosted-panel, or classic. Hosted-panel requires explicit selection.
CHAT_BACKEND UI v2.6.0 treats missing or blank as hosted_agent; an umbrella deployment must publish the resolved sticky value. orchestrator is the explicit fallback. Unknown values fail startup. Environment configuration takes precedence over App Configuration.
ORCHESTRATOR_BASE_URL Classic service root, used only when CHAT_BACKEND=orchestrator. The UI calls the /orchestrator route on this endpoint.
HOSTED_AGENT_BASE_URL Required HTTPS hosted service root. Orchestrator v4.0.0 defines stateless POST /responses; UI v2.6.0 currently sends complete ordered messages through the distinct POST /invocations compatibility route. Continuity remains off until the live call route satisfies the protocol evidence gate.
HOSTED_AGENT_RESOURCE_SCOPE Required explicit non-ARM hosted data-plane Entra scope ending in /.default, for example api://<application-id>/.default.
HOSTED_AGENT_AUTH_MODE user_delegated is the default and required continuity path. Under OQ-OWN, it means the trusted UI BFF derives x-ms-user-identity; it does not mean an OBO token is sent to the agent. OBO remains a separate retrieval flow. service_identity is an explicit reviewed exception that is incompatible with owner-bound continuity, so continuity stays off/503 in that mode.
HOSTED_AGENT_SSE_IDLE_TIMEOUT_SECONDS Finite positive wait for the next SSE event. The UI default is 60; an infinite timeout is rejected.
HOSTED_AGENT_IMAGE_VERSION Canonical lowercase immutable digest in sha256:<64-hex-characters> form. Mutable tags are rejected.
SEARCH_SERVICE_UAI_RESOURCE_ID Required identity boundary for private Search. Post-provisioning preserves an explicit value or resolves the single Search user-assigned identity from the Search resource; it must not publish an empty replacement.
OTEL_INSTRUMENTATION_GENAI_CAPTURE_MESSAGE_CONTENT Generative-AI prompt and completion telemetry capture. Defaults to false. Set to true only when the deployment's data-handling policy explicitly permits sensitive content telemetry.

Hosted configuration, authentication, connection, timeout, protocol, and runtime failures are terminal for startup or the affected request. The UI does not silently switch to the orchestrator or to managed identity. Foundry passes opaque x-agent-foundry-call-id context to Toolbox; user and delegated bearer tokens are not copied into tool payloads or client-defined identity headers.

Hosted conversation continuity platform gate

Delegated owner binding remains evidence-gated

HOSTED_CONTINUITY_ENABLED defaults to false and must stay false. The exact component releases are pinned, but deployment must still prove the live 100%-routed Responses protocol 2.0.0, trusted UI BFF identity derivation, and the two exact direct agent-scoped roles before recording HOSTED_CONVERSATION_OWNER_BINDING_VALIDATED=true. Otherwise compatible history endpoints return HTTP 503.

The trusted UI BFF derives x-ms-user-identity from the authenticated server-side principal and sends it on the hosted Responses request. This owner header is not an OBO token: OBO remains a separate downstream retrieval flow with its own audience and bearer token.

Activation assigns only the UI BFF:

  • built-in Foundry Agent Consumer (eed3b665-ab3a-47b6-8f48-c9382fb1dad6), whose only DataAction is Microsoft.CognitiveServices/accounts/AIServices/endpoints/interact/action; and
  • custom GPT-RAG Hosted Agent User Identity Impersonation (bef66abe-a495-530a-be1d-5d882fecff03), containing only the reviewed Microsoft.CognitiveServices/accounts/AIServices/agents/endpoints/UserIdentityImpersonation/action DataAction.

Both assignments must be direct ServicePrincipal assignments scoped to the individual hosted agent. Broader, inherited, group-derived, wildcard, custom-equivalent, or extra-DataAction access fails validation. Foundry User and Project Runtime User are prohibited substitutes. The hosted runtime is not an identity-header source and receives no key, Conversation or impersonation RBAC, or Cosmos DB in hosted/no-panel.

Setting or gate Required posture
HOSTED_CONTINUITY_ENABLED false until delegated owner binding validates; false/missing validation means history HTTP 503.
HOSTED_CONVERSATION_OWNER_BINDING Defaults to delegated; capability is the only accepted explicit fallback.
HOSTED_CONVERSATION_OWNER_BINDING_VALIDATED Becomes true only after the live protocol, identity-source, role-definition, assignment, and scope checks pass.
HOSTED_CONVERSATION_DELEGATED_IDENTITY_HEADER Must be exactly x-ms-user-identity.
HOSTED_CONVERSATION_DELEGATED_IDENTITY_SOURCE Must be exactly authenticated_ui_bff_principal.
HOSTED_CONVERSATIONS_TOKEN_AUDIENCE Exact Foundry audience https://ai.azure.com; distinct from x-ms-user-identity and downstream OBO audiences.
HOSTED_AGENT_RESPONSES_PROTOCOL_VERSION Must be exactly 2.0.0.
HOSTED_CONTINUITY_UNAVAILABLE_STATUS_CODE Must be 503.
HOSTED_HISTORY_MAX_ITEMS Default 100; accepted range 1-1,000.
HOSTED_HISTORY_MAX_TOKENS Default 32000; accepted range 1-1,000,000.
HOSTED_HISTORY_TRUNCATION Must be drop_oldest.

Capability/HMAC is a disabled fallback only. The delegated primary path does not create a capability key, require HOSTED_CONTINUITY_KEY_VAULT_URI/HOSTED_CONTINUITY_KEY_VAULT_NAME, publish HOSTED_CONVERSATION_CAPABILITY_KEY, or grant a capability-secret role. Fallback key ID, TTL, vault, reference, and retained key-history behavior apply only if a future release explicitly selects and validates capability mode. See the hosted conversation continuity platform contract for the complete trust and rollout boundary.

POST /responses and POST /invocations are distinct protocols, not aliases, and their request bodies are not interchangeable. Orchestrator v4.0.0 implements a stateless hosted Responses contract: callers send complete ordered text history in input on every request. A non-empty string is valid for one turn. An array must be non-empty, text-only, ordered oldest to newest, and end with a non-empty user message. Top-level conversation and previous_response_id are rejected with HTTP 422.

{
  "input": [
    {
      "role": "user",
      "content": "What is the document retention policy?"
    },
    {
      "role": "assistant",
      "content": "The policy states 30 days."
    },
    {
      "role": "user",
      "content": "Who approves an exception?"
    }
  ],
  "stream": true,
  "store": false
}

The compatibility POST /invocations route retains the legacy messages-based schema:

{
  "messages": [
    {
      "role": "user",
      "content": "What is the document retention policy?"
    }
  ],
  "conversation_id": "<conversation-id>",
  "metadata": {}
}

The hosted runtime constructs no managed-Conversations client and performs zero managed state operations. The compatibility conversation_id is an opaque label for tagging and local retrieval scoping only; it is not an ownership credential. Managed history, user list/read/feedback/delete, and opaque handle validation are owned by the trusted UI BFF. See the hosted-agent release matrix.

Two-phase hosted deployment

For a fresh hosted/no-panel deployment, configure the delegated data-plane scope before the first provision:

azd env set HOSTED_AGENT_RESOURCE_SCOPE "api://<application-id>/.default"
azd env set HOSTED_AGENT_SSE_IDLE_TIMEOUT_SECONDS 60
# Keep continuity disabled until live protocol, identity, and role evidence validates ownership.
azd env set HOSTED_CONTINUITY_ENABLED false
azd provision
pwsh scripts/prepareHostedDeployment.ps1
azd provision
azd deploy

On POSIX systems, use scripts/prepareHostedDeployment.sh for the preparation step. For an explicit migration, set DEPLOYMENT_TOPOLOGY=hosted-no-panel before the first azd provision.

For the supported hosted-panel topology, use the same two-phase flow with:

azd env set DEPLOYMENT_TOPOLOGY hosted-panel
azd env set HOSTED_CONTINUITY_ENABLED false

This composes UI and ingestion plus only the two panel metadata containers. PANEL_HISTORY_ENABLED, PANEL_HISTORY_OWNER_BINDING_VALIDATED, and PANEL_OPERATOR_SURFACES_ENABLED remain deployment-published false; do not override them before their separate evidence and authorization procedures complete. The corresponding routes return HTTP 503 while disabled.

Panel post-provisioning resolves exactly one managed-identity principal from the frontend Container App and exactly one from ingestion. It then creates only container-scoped Cosmos SQL grants on panel-conversation-owner-index and panel-feedback: Cosmos DB Built-in Data Contributor for frontend and Cosmos DB Built-in Data Reader for ingestion. Missing or ambiguous Container App identities fail setup. Do not substitute account-scope grants, grant ingestion write access, or grant the hosted agent any panel Cosmos role.

The first provision creates hosted prerequisites with image preparation enabled but hosted deployment disabled. The preparation command clones and verifies the manifest-pinned orchestrator source, builds the standard image and the hosted-entrypoint derivative, resolves the pushed manifest to an immutable digest, and persists that digest and source provenance. The second provision materializes the digest-backed hosted handoff; azd deploy then deploys the hosted agent.

Public deployments use shared ACR Tasks. Network-isolated deployments use the dedicated VNet-connected ACR Tasks agent pool; shared ACR Tasks cannot reach a private endpoint. Operators may pass an already-built immutable sha256:<64-hex-characters> digest to the preparation command to skip builds. No lifecycle hook recursively invokes azd provision.

The child hosted-agent/azure.yaml service definition is part of the prebuilt handoff contract. It must declare language: docker and docker.remoteBuild: true. The parent pre-deploy hook sets AZD_AGENT_SKIP_ACR=true in the child azd environment before azd deploy orchestrator-agent, so the already-prepared immutable image is used instead of triggering another ACR build. Do not remove any of these three settings from a prebuilt hosted deployment.

Current runtime readiness blocker

The latest implementation validation activated the agent version, but a new session readiness request returned HTTP 424. This is not a successful hosted runtime validation. Keep continuity and panel evidence gates false/off/503, keep the classic rollback available, and do not describe the integrated matrix as shipped until readiness and the remaining live checks pass.

Explicit classic fallback

Fallback is a deployment operation, not a request-time retry:

azd env set DEPLOYMENT_TOPOLOGY classic
azd provision
azd deploy

This restores the orchestrator Container App and publishes CHAT_BACKEND=orchestrator without deleting hosted Conversations or existing classic panel data.

Current classic release

GPT-RAG v3.7.0 remains the latest published umbrella release at the time this matrix was documented. Its classic pin set is UI v2.3.13, orchestrator v3.8.0, ingestion v2.5.0, and AI Landing Zone v2.3.0. The exact hosted matrix is pinned by the current integration manifest, whose umbrella tag remains unreleased; do not mix its topology/configuration contract into the older v3.7.0 hooks or manifest.

Retrieval backend

GPT-RAG can retrieve grounding content directly from Azure AI Search or through a Foundry IQ knowledge base. Starting with GPT-RAG v3.0.2 and AI Landing Zone v2.1.2, new deployments use Foundry IQ by default through a native Azure Blob Knowledge Source. Existing deployments can stay on RETRIEVAL_BACKEND=ai_search until you explicitly migrate.

Use the grounding sources overview to understand the default Foundry IQ path, when to keep using Azure AI Search, and when to use the searchIndex pattern for custom GPT-RAG ingestion pipelines.

The most important settings are:

Setting Typical value Purpose
RETRIEVAL_BACKEND foundry_iq for new deployments, ai_search for existing compatibility or rollback Selects the retrieval path.
FOUNDRY_IQ_PATTERN azureBlob by default, or searchIndex for custom GPT-RAG ingestion Selects the Foundry IQ setup choice.
KNOWLEDGE_BASE_NAME <env>-knowledge-base Foundry IQ knowledge base name.
KNOWLEDGE_BASE_CONNECTION_ID Generated by AILZ Dedicated Foundry connection for knowledge-base use.
FOUNDRY_IQ_API_VERSION 2026-05-01-preview Required for per-user permissions and custom ingestion path filterAddOn.
FOUNDRY_IQ_KNOWLEDGE_RETRIEVAL_BILLING_PLAN free or standard Controls Azure AI Search agentic retrieval billing.

With the default Blob path, Foundry IQ processes files directly from the documents container. GPT-RAG ingestion is not used in that path. Use FOUNDRY_IQ_PATTERN=searchIndex only when you intentionally keep a custom GPT-RAG ingestion pipeline that writes chunks to Azure AI Search.

Two optional Foundry IQ Knowledge Sources can run alongside the documents source on the same Knowledge Base. Both are off by default and require signed-in users:

Demo video:

Zero Trust Deployment

For deployments that require network isolation.

Network-isolated deployments use a two-host flow:

Phase Where to run Command
Provision infrastructure Workstation azd provision
Configure data-plane resources Jumpbox or VNet-connected host scripts/postProvision.ps1
Deploy services Jumpbox or VNet-connected host azd deploy

Do not run azd deploy from the workstation when NETWORK_ISOLATION=true. The deploy hook blocks that path because private resources and the private ACR build pool are reachable only from inside the VNet.

Network Isolation runbook

Use this runbook for a clean network-isolated deployment:

  1. On your workstation, create or select the azd environment and enable network isolation.
  2. Still on your workstation, run azd provision. This creates the infrastructure and then stops before local data-plane configuration.
  3. Connect to the jumpbox through Azure Bastion, or use another machine with VNet/VPN access.
  4. On the jumpbox, authenticate with the VM managed identity.
  5. On the jumpbox, run scripts/postProvision.ps1 with RUN_FROM_JUMPBOX=true.
  6. On the jumpbox, run azd deploy with RUN_FROM_JUMPBOX=true. To build the UI, orchestrator, ingestion, and hosted-agent derivative images with the dedicated VNet-connected ACR Tasks agent pool, set ACR_TASK_AGENT_POOL=build-pool.

BUILD_MODE is normally not required when deploying the UI, orchestrator, or ingestion services. The hosted-agent derivative image can use the same dedicated pool. Shared ACR Tasks cannot reach a private endpoint.

Regional preflight

Run preflight before every Zero Trust deployment. It is much faster to fail in the first few minutes than to wait for a long network-isolated deployment and then discover that a regional dependency cannot be created.

azd provision runs the scripts/preProvision hook. The hook invokes scripts/Invoke-RegionalPreflight.ps1 before the Azure Resource Manager deployment starts.

Preflight checks include:

  • the selected Azure region and provider support,
  • common regional readiness checks for Azure AI Search, Cosmos DB, Container Apps, AI Foundry, and Cognitive Services,
  • jumpbox VM SKU availability and restrictions,
  • Azure OpenAI model quota for the configured deployments.

Preflight is an early warning, not a live capacity reservation. Azure capacity can still change after the check passes, and some regional capacity errors are only returned when Azure creates the resource. Recent examples include Azure AI Search Standard capacity in Sweden Central and Cosmos DB zonal capacity in West Europe.

Use the result this way:

Result Operator action
FAIL Stop. Fix the subscription, quota, region, or parameter issue before provisioning.
WARN Review the warning before continuing. If it mentions capacity or regional risk, consider changing region first.
Pass Continue, but keep the deployment logs open because live capacity can still change.

If a region fails or warns on a critical dependency, try another fully supported region instead of waiting 30 minutes or more for a deployment that is likely to fail. Use GPT_RAG_REGIONAL_PREFLIGHT_SKIP=true only when you intentionally bypass regional checks, or PREFLIGHT_SKIP=true to bypass all preflight hooks.

Before Provisioning

Enable network isolation in your environment:

azd env set NETWORK_ISOLATION true

Optional v2 parameters can be set before provisioning:

azd env set DEPLOYMENT_MODE standalone
azd env set VM_SIZE Standard_D2s_v3
azd env set ENABLE_COSMOS_ANALYTICAL_STORAGE false

ALLOWED_IP_RANGES is also available for CIDR allow-listing, but because it is an array parameter, prefer editing main.parameters.json or using a parameter overlay rather than storing a complex array in the azd environment.

Make sure you’re signed in with your Azure user account:

az login
azd auth login

Add --tenant for az or --tenant-id for azd if you want a specific tenant.

Provision Infrastructure

azd env set AZURE_SKIP_NETWORK_ISOLATION_WARNING true   # optional for automation; skips the local post-provision prompt
azd provision

Post-Provision Configuration

With NETWORK_ISOLATION=true, data-plane configuration must run from inside the VNet. A workstation should only run azd provision; if it does not have VNet/VPN access, the local post-provision hook will skip data-plane work and tell you to continue from the jumpbox.

Using the Jumpbox VM

1) Reset the VM password in the Azure Portal (required on first access if not set in deployment parameters):

  • Go to your VM resource → Support + troubleshooting → Reset password → Set new credentials
  • Default username is testvmuser

2) Connect via Azure Bastion

3) Authenticate with the VM's Managed Identity:

az login --identity
azd auth login --managed-identity

Add --tenant for az or --tenant-id for azd if you want a specific tenant.

4) Run the post-provision script:

PowerShell: 

cd C:\github\GPT-RAG
azd env set RUN_FROM_JUMPBOX true
.\scripts\postProvision.ps1

Bash: 

cd /mnt/c/github/gpt-rag
./scripts/postProvision.sh

Note: If you have re-initialized or cloned the gpt-rag repo again, refresh your azd environment before running the postProvision script so it points to the existing deployment: azd init -t azure/gpt-rag then azd env refresh. When prompted, select the same Subscription, Resource Group, and Location as the original provisioning so azd correctly links to your environment.

Existing Platform / AI Landing Zone Integrated

Use these settings when GPT-RAG must deploy into an existing enterprise platform, such as a hub-spoke network with centrally managed Private DNS Zones, Log Analytics, Application Insights, Bastion, NAT Gateway, or Azure Firewall.

Core mode: set DEPLOYMENT_MODE to ailz-integrated, then pass the existing resource IDs that your platform team owns. The default remains standalone, so basic deployments do not require these settings.

azd env set DEPLOYMENT_MODE ailz-integrated
azd env set USE_EXISTING_VNET true
azd env set EXISTING_VNET_RESOURCE_ID "/subscriptions/<sub>/resourceGroups/<rg>/providers/Microsoft.Network/virtualNetworks/<vnet>"

Existing Private DNS Zones: set the zone resource IDs for services already managed by the platform. Common values include EXISTING_PRIVATE_DNS_ZONE_OPENAI_RESOURCE_ID, EXISTING_PRIVATE_DNS_ZONE_AISERVICES_RESOURCE_ID, EXISTING_PRIVATE_DNS_ZONE_SEARCH_RESOURCE_ID, EXISTING_PRIVATE_DNS_ZONE_COSMOS_RESOURCE_ID, EXISTING_PRIVATE_DNS_ZONE_BLOB_RESOURCE_ID, EXISTING_PRIVATE_DNS_ZONE_KEYVAULT_RESOURCE_ID, EXISTING_PRIVATE_DNS_ZONE_APPCONFIG_RESOURCE_ID, EXISTING_PRIVATE_DNS_ZONE_CONTAINERAPPS_RESOURCE_ID, EXISTING_PRIVATE_DNS_ZONE_ACR_RESOURCE_ID, and Azure Monitor / App Insights zone IDs.

azd env set EXISTING_PRIVATE_DNS_ZONE_SEARCH_RESOURCE_ID "/subscriptions/<sub>/resourceGroups/<dns-rg>/providers/Microsoft.Network/privateDnsZones/privatelink.search.windows.net"
azd env set EXISTING_PRIVATE_DNS_ZONE_OPENAI_RESOURCE_ID "/subscriptions/<sub>/resourceGroups/<dns-rg>/providers/Microsoft.Network/privateDnsZones/privatelink.openai.azure.com"
azd env set DNS_ZONE_LINK_SUFFIX "<unique-spoke-name>"

Shared platform resources: set EXISTING_LOG_ANALYTICS_WORKSPACE_RESOURCE_ID, EXISTING_APPLICATION_INSIGHTS_RESOURCE_ID, EXISTING_APPLICATION_INSIGHTS_CONNECTION_STRING, HUB_INTEGRATION_HUB_VNET_RESOURCE_ID, HUB_INTEGRATION_EGRESS_NEXT_HOP_IP, or HUB_INTEGRATION_EXISTING_ROUTE_TABLE_RESOURCE_ID when those resources are centrally managed.

Spoke resource switches: use DEPLOY_JUMPBOX, DEPLOY_BASTION, DEPLOY_NAT_GATEWAY, EXISTING_JUMPBOX_RESOURCE_ID, EXISTING_BASTION_RESOURCE_ID, EXISTING_NAT_GATEWAY_RESOURCE_ID, DEPLOY_AZURE_FIREWALL, and DEPLOY_ACR_TASK_AGENT_POOL to align GPT-RAG with the platform topology. These are optional and preserve the default standalone behavior when unset.

Deploy GPT-RAG Services

Note: For Zero Trust deployments with network isolation, deploy services from the jumpbox or another host with VNet connectivity. If using the jumpbox VM, the repositories are located in the C:\github directory.

Once the GPT-RAG infrastructure is provisioned, you can deploy the services.

To deploy all services at once, navigate to the gpt-rag directory (with azd environment configured) and run:

cd C:\github\GPT-RAG
azd env set RUN_FROM_JUMPBOX true
azd env set NETWORK_ISOLATION true
azd env set ACR_TASK_AGENT_POOL build-pool
azd deploy

This command deploys the services selected by the active mode. The UI, orchestrator, ingestion, and hosted-agent derivative images can use Azure Container Registry remote builds (az acr build) against the dedicated VNet-connected ACR Tasks agent pool provided by AILZ v2.4.1. Shared ACR Tasks cannot reach a private endpoint.

The deploy hook uses NETWORK_ISOLATION as the source of truth. When NETWORK_ISOLATION=true, azd deploy fails fast unless it is running from the VNet with RUN_FROM_JUMPBOX=true. The older AZURE_ZERO_TRUST variable is not used.

If you prefer to deploy a single service, for example, when updating only that service, you can deploy it individually. Below is an example using the orchestrator service. The same approach applies to other services (frontend, dataingest, mcp).

Deploy Individual Services

Make sure you're logged in to Azure:

az login

Example: Deploying the Orchestrator

Using azd (recommended):

Initialize the template: 

azd init -t azure/gpt-rag-orchestrator

Important: Use the same environment name with azd init as in the infrastructure deployment to keep components consistent.

Update environment variables then deploy: 

azd env refresh
azd deploy

Important: Run azd env refresh with the same subscription and resource group used in the infrastructure deployment.

Using a shell script:

Clone the repository, set the App Configuration endpoint, and run the deployment script.

PowerShell (Windows): 

git clone https://github.com/Azure/gpt-rag-orchestrator.git
$env:APP_CONFIG_ENDPOINT = "https://<your-app-config-name>.azconfig.io"
cd gpt-rag-orchestrator
.\scripts\deploy.ps1

Bash (Linux/macOS): 

git clone https://github.com/Azure/gpt-rag-orchestrator.git
export APP_CONFIG_ENDPOINT="https://<your-app-config-name>.azconfig.io"
cd gpt-rag-orchestrator
./scripts/deploy.sh

Permissions

The role tables below describe the classic Container Apps topology. The pinned hosted matrix omits the orchestrator Container App assignments and adds Foundry data-plane, delegated-user, Toolbox, and immutable-image pull assignments. Hosted-panel adds only the narrow panel metadata-container assignments; its user and operator surfaces remain off/503 behind independent evidence gates.

Microsoft Foundry Role and AI Search Assignments

Resource Role Assignee Description
GenAI App Search Service Search Index Data Reader Microsoft Foundry Project Read index data
GenAI App Search Service Search Service Contributor Microsoft Foundry Project Create AI Search connection
GenAI App Storage Account Storage Blob Data Reader Microsoft Foundry Project Read blob data
Microsoft Foundry Account Cognitive Services User Search Service Allow Search Service to access vectorizers

Container App Role Assignments

Resource Role Assignee Description
GenAI App Configuration Store App Configuration Data Reader ContainerApp: orchestrator Read configuration data
GenAI App Configuration Store App Configuration Data Reader ContainerApp: frontend Read configuration data
GenAI App Configuration Store App Configuration Data Reader ContainerApp: dataingest Read configuration data
GenAI App Configuration Store App Configuration Data Reader ContainerApp: mcp Read configuration data
GenAI App Container Registry AcrPull ContainerApp: orchestrator Pull container images
GenAI App Container Registry AcrPull ContainerApp: frontend Pull container images
GenAI App Container Registry AcrPull ContainerApp: dataingest Pull container images
GenAI App Container Registry AcrPull ContainerApp: mcp Pull container images
GenAI App Key Vault Key Vault Secrets User ContainerApp: orchestrator Read secrets
GenAI App Key Vault Key Vault Secrets User ContainerApp: frontend Read secrets
GenAI App Key Vault Key Vault Secrets User ContainerApp: dataingest Read secrets
GenAI App Key Vault Key Vault Secrets User ContainerApp: mcp Read secrets
GenAI App Search Service Search Index Data Reader ContainerApp: orchestrator Read index data
GenAI App Search Service Search Index Data Contributor ContainerApp: dataingest Read/write index data
GenAI App Search Service Search Index Data Contributor ContainerApp: mcp Read/write index data
GenAI App Storage Account Storage Blob Data Reader ContainerApp: orchestrator Read blob data
GenAI App Storage Account Storage Blob Data Reader ContainerApp: frontend Read blob data
GenAI App Storage Account Storage Blob Data Contributor ContainerApp: dataingest Read/write blob data
GenAI App Storage Account Storage Blob Data Contributor ContainerApp: mcp Read/write blob data
GenAI App Cosmos DB Cosmos DB Built-in Data Contributor ContainerApp: orchestrator Read/write Cosmos DB data
Microsoft Foundry Account Cognitive Services User ContainerApp: orchestrator Access Cognitive Services
Microsoft Foundry Account Cognitive Services User ContainerApp: dataingest Access Cognitive Services
Microsoft Foundry Account Cognitive Services User ContainerApp: mcp Access Cognitive Services
Microsoft Foundry Account Cognitive Services OpenAI User ContainerApp: orchestrator Use OpenAI APIs
Microsoft Foundry Account Cognitive Services OpenAI User ContainerApp: dataingest Use OpenAI APIs
Microsoft Foundry Account Cognitive Services OpenAI User ContainerApp: mcp Use OpenAI APIs

Executor Role Assignments

Resource Role Assignee Description
GenAI App Configuration Store App Configuration Data Owner Executor Full control over configuration settings
GenAI App Container Registry AcrPush Executor Push container images
GenAI App Container Registry AcrPull Executor Pull container images
GenAI App Key Vault Key Vault Contributor Executor Manage Key Vault settings
GenAI App Key Vault Key Vault Secrets Officer Executor Create Key Vault secrets
GenAI App Search Service Search Service Contributor Executor Create/update search service elements
GenAI App Search Service Search Index Data Contributor Executor Read/write search index data
GenAI App Search Service Search Index Data Reader Executor Read index data
GenAI App Storage Account Storage Blob Data Contributor Executor Read/write blob data
GenAI App Cosmos DB Cosmos DB Built-in Data Contributor Executor Read/write Cosmos DB data
Microsoft Foundry Account Cognitive Services OpenAI User Executor Use OpenAI APIs

Jumpbox VM Role Assignments

Resource Role Assignee Description
GenAI App Container Apps Container Apps Contributor Jumpbox VM Full control over Container Apps
Azure Managed Identity Managed Identity Operator Jumpbox VM Assign and manage user-assigned identities
GenAI App Container Registry Container Registry Repository Writer Jumpbox VM Write to ACR repositories
GenAI App Container Registry Container Registry Tasks Contributor Jumpbox VM Manage ACR tasks
GenAI App Container Registry Container Registry Data Access Configuration Administrator Jumpbox VM Manage ACR data access configuration
GenAI App Container Registry AcrPush Jumpbox VM Push container images
GenAI App Configuration Store App Configuration Data Owner Jumpbox VM Full control over configuration settings
GenAI App Key Vault Key Vault Contributor Jumpbox VM Manage Key Vault settings
GenAI App Key Vault Key Vault Secrets Officer Jumpbox VM Create Key Vault secrets
GenAI App Search Service Search Service Contributor Jumpbox VM Create/update search service elements
GenAI App Search Service Search Index Data Contributor Jumpbox VM Read/write search index data
GenAI App Storage Account Storage Blob Data Contributor Jumpbox VM Read/write blob data
GenAI App Cosmos DB Cosmos DB Built-in Data Contributor Jumpbox VM Read/write Cosmos DB data
Microsoft Foundry Account Cognitive Services Contributor Jumpbox VM Manage Cognitive Services resources
Microsoft Foundry Account Cognitive Services OpenAI User Jumpbox VM Use OpenAI APIs
© 2025 GPT-RAG — powered by ❤️ and coffee ☕