Azure AI Gateway Labs
Documentation Open portal
Governance & cost · Lab 08

Screen prompts and responses with content safety

Add a content safety policy that screens prompts, tool inputs, and responses for hate, self-harm, sexual, and violent content, tune each category's threshold, and see how blocked calls return 400.

Beginner 20 min PortalCLIRESTBicepTerraform Updated
Source Edit

What you'll do

  • Explain how content safety severity thresholds decide what the gateway blocks
  • Add a content safety policy to a model in the portal, with the Azure CLI, or with the REST API
  • Override the threshold for one category without changing the others
  • Publish a screened alias with Bicep and azd or with Terraform, without touching existing clients
  • Test the policy with benign prompts and tell a gateway block from a Foundry content filter result

Before you start

Tracks: Portal, Azure CLI, REST, Bicep + azd, Terraform

Your gateway now serves a customer-facing assistant and several internal agents, on models from more than one provider. Your responsible AI review asks for one consistent rule: every prompt and every response is screened for harmful content, whichever model or tool handles it. In this lab you add a content safety policy at the gateway, tune it per category, and see what a blocked call looks like to a client.

Before you begin

One-time setupThe Setup guide tasks that this lab uses. Mark each one done once, and the labs stop listing it.

Tip

On the Bicep + azd and Terraform tracks, azd up or terraform apply creates its own gateway, model, and key, so you can skip this section. You need only the one-time setup above.

  • Complete Publish a Foundry model and make your first call. You need a gateway, an imported model, a callable alias, and a runtime API key from AccessAPI keys.
  • Permission to change the gateway, for example Contributor on the resource group.
  • Run the scripts and samples from the lab's folder, ~/aigateway/labs/08-content-safety.

Set these variables once. Every track and the Verify section reuse them.

# Name your gateway, its model provider, and the backend model that gets the policy
export RG="<resource-group>"
export GATEWAY="<gateway>"                 # 24 characters or fewer
export PROVIDER="<provider-name>"          # for example foundry-models
export MODEL="<model-name>"                # the backend model name under that provider

# Read the gateway's runtime URL, which apps call
export AI_GATEWAY_URL="https://$(az aigateway show -g "$RG" -n "$GATEWAY" --query properties.frontend.defaultHostname -o tsv)"
echo "🌐 Runtime URL: $AI_GATEWAY_URL"

# Read the alias that apps send in the model field: for Foundry imports, it matches the deployment's model name
export AI_GATEWAY_MODEL="$(az aigateway model show -g "$RG" --gateway-name "$GATEWAY" --provider-name "$PROVIDER" \
  -n "$MODEL" --query properties.deployment.modelName -o tsv)"
echo "🔖 Model alias: $AI_GATEWAY_MODEL"

# Paste the runtime key you created in Access › API keys, without showing it: don't commit it or write it to a file
printf "AI Gateway API key: " && read -rs AI_GATEWAY_API_KEY && export AI_GATEWAY_API_KEY && echo
echo "🔑 Key: in AI_GATEWAY_API_KEY (${#AI_GATEWAY_API_KEY} characters, not shown)"

How it works

A content safety policy screens prompts, tool inputs, and responses with Azure AI Content Safety. It rates content in four categories: Hate, Self-harm, Sexual, and Violence. You set a severity threshold for each category independently. When content reaches the threshold, the gateway blocks the call and returns 400. A blocked prompt never reaches the backend.

The threshold is the lowest severity that gets blocked, so a lower threshold blocks more:

Portal labelCLI and REST valueWhat it blocks
StrictLowThe most content: low, medium, and high severity
Balanced (default)MediumMedium and high severity
LenientHighOnly the most concerning, high-severity content
OffNoneNothing in this category

You can attach content safety to models, aliases, and MCP servers, or set it gateway-wide so that it covers every compatible asset, including ones added later. On an MCP server, it screens the tool inputs that agents send and the results that tools return. Every applicable policy is evaluated, and any one of them can block the call.

Note

Foundry deployments have their own Content filter, shown on the model's details page in Models. That filter runs inside Foundry, for that one deployment. The gateway policy runs before the backend, applies the same rules to models from any provider and to MCP tool traffic, and is managed by the platform team in one place. Keep both: a request has to pass each of them.

Add content safety to the model

Pick one track. Each track produces the same policy: Balanced (Medium) for Hate, Self-harm, and Sexual, and Strict (Low) for Violence. The portal, CLI, and REST tracks put it on your model. The Bicep + azd and Terraform tracks create a new gateway and put it on a new alias of the model. The stricter Violence threshold makes the test in Verify more likely to show a block, using only a mild, non-graphic prompt.

Choose your track
1

Create the policy

The Add policy wizard doesn't list Content safety for AI Gateway resources, so create the policy with the Azure CLI, for example in Azure Cloud Shell, with the variables from Before you begin. Then set the thresholds in the portal. The commands read the model's policies, add Balanced (Medium) content safety on all four categories, and write the array back, so every other policy on the model stays:

# Read the model's policies, drop any content safety policy, and add one at Medium (Balanced) in all four categories
POLICIES="$(az aigateway model policy list -g "$RG" --gateway-name "$GATEWAY" --provider-name "$PROVIDER" -n "$MODEL" -o json |
  jq -c '[.[] | select(.type != "contentSafety")]
    + [{"type":"contentSafety","hateSeverity":"Medium","violenceSeverity":"Medium","sexualSeverity":"Medium","selfHarmSeverity":"Medium"}]')"

# Write the merged array back: --policies replaces the whole array
az aigateway model update -g "$RG" --gateway-name "$GATEWAY" --provider-name "$PROVIDER" -n "$MODEL" \
  --policies "$POLICIES" --output none &&
  echo "🔐 Content safety on $MODEL is Balanced (Medium) for all four categories"
2

Set the thresholds

Open the AI Gateway portal and select your gateway from the gateway switcher. Select Policies. The list shows a Content safety policy with your model under Scope and Balanced filtering under Configuration. Select Balanced filtering to open the policy, then select Edit.

In the Edit policy panel, choose how strictly each category is filtered:

FieldValue
HateBalanced
ViolenceStrict
Self-harmBalanced
SexualBalanced
Edit policy sets a level for each category: Strict, Balanced, Lenient, or Off.

Select Save. The portal confirms Policy updated.

Checkpoint

Under Configuration, the policy shows Hate Filtering: Balanced, Violence Filtering: Strict, Sexual Filtering: Balanced, and Self Harm Filtering: Balanced, with your model under Applies to. In Policies, its configuration now reads Strict filtering, the strictest level among the categories.

Verify

Test the policy with benign prompts only. You don't need harmful content to see how the gateway behaves, and you should never write it to test a filter.

1

Run the test prompts

From the lab folder, run the script for your shell. It sends two prompts: a neutral question, and a request for a short, non-graphic scene from a historical novel. It prints the status and the start of the response or the error for each one.

# Send a neutral prompt and a mild fiction prompt through the screened alias
echo "🚦 Expect 200 for the neutral prompt, and 200 or 400 for the fiction prompt:"
./infra/test-content-safety.sh "$AI_GATEWAY_MODEL"

On GitHub, open test-content-safety.sh or test-content-safety.ps1.

Example output:

Text
🚦 Expect 200 for the neutral prompt, and 200 or 400 for the fiction prompt:
Prompt    Status  Finish reason   Response or error
neutral   200     stop            Teams put an API gateway in front of AI models to centralize access co
fiction   200     stop            The two armies massed on either side of the muddy approach, banners sn

The neutral prompt returns 200. The fiction prompt can go either way, and your results will vary: Content Safety rates both the prompt and the model's response, and a mild battle scene sits close to the Strict Violence threshold. If it returns 400, the gateway blocked it and the error message says why. If it returns 200, the content was rated below the threshold.

Note

The Foundry deployment's own filter can act too. A 400 that starts with "The response was filtered due to the prompt triggering Azure OpenAI's content management policy", or a 200 with the finish reason content_filter, comes from Foundry, not from the gateway. At the time of writing, AI Gateway resources save the content safety policy but don't block calls with it yet, so only Foundry results show up, even at Strict. Until the gateway enforces the policy, rely on the deployment's content filter, as described in Check the Foundry content filter, and see Troubleshooting.

Important

If the fiction prompt passes, don't make it more graphic to force a block. The point of this test is to understand thresholds with safe input, not to probe the filter. Use Monitoring and real, representative traffic in a test environment to tune thresholds.

2

Relax one category

Set Violence back to Balanced (Medium) with the track you used, then run the script again. For example, from the command line, set all four categories to Medium. The other three already are, so only Violence changes, and the merge keeps every other policy on the model:

# Set all four categories to Medium (Balanced): only Violence changes, and the other policies stay
POLICIES="$(az aigateway model policy list -g "$RG" --gateway-name "$GATEWAY" --provider-name "$PROVIDER" -n "$MODEL" -o json |
  jq -c '[.[] | select(.type != "contentSafety")]
    + [{"type":"contentSafety","hateSeverity":"Medium","violenceSeverity":"Medium","sexualSeverity":"Medium","selfHarmSeverity":"Medium"}]')"

# Write the merged array back: --policies replaces the whole array
az aigateway model update -g "$RG" --gateway-name "$GATEWAY" --provider-name "$PROVIDER" -n "$MODEL" \
  --policies "$POLICIES" --output none &&
  echo "🔐 Content safety on $MODEL is Balanced (Medium) for all four categories"

On the Bicep + azd track, set the environment value and provision again instead:

# Relax Violence to Medium (Balanced) in the azd environment
azd env set CONTENT_SAFETY_VIOLENCE Medium &&
  echo "📝 CONTENT_SAFETY_VIOLENCE is Medium"

# Provision again: only the screened alias changes
azd provision &&
  echo "🔐 Content safety on $AI_GATEWAY_MODEL is Balanced (Medium) for all four categories"

On the Terraform track, in the lab folder, run terraform -chdir=terraform plan -var violence_severity=Medium -out tfplan, then terraform -chdir=terraform apply tfplan. The plan ends with Plan: 0 to add, 1 to change, 0 to destroy., for azapi_resource.safe_alias.

A prompt that Strict blocked is more likely to pass at Balanced. Prompts that passed before still pass.

3

Check the Foundry content filter

Open ModelsBackend models and select your model. On its Overview tab, Content filter under Model deployment names the Foundry deployment's own filter, for example Microsoft.DefaultV2. It's separate from the gateway policy. When Foundry's filter acts instead of the gateway, the error or the finish_reason comes from the backend, and it can mention content_filter.

4

Find blocked calls in Monitoring

If you configured Application Insights in Monitor usage, cost, and traces, open Monitoring and look for 400 outcomes on your model. Review them before and after every threshold change.

Checkpoint

The neutral prompt returned 200, you saw how the borderline prompt fared at Strict and at Balanced, and you can tell a gateway block (400 from the content safety policy) from a Foundry content filter result.

Go further

  • Screen tool traffic. Add a Content safety policy to the learn MCP server from Federate MCP tools behind one endpoint. With REST, use the same contentSafety object on .../workspaces/default/toolServers/learn.
  • Make it the default. Apply Balanced content safety to every model and MCP server, including future ones, in Set gateway-wide policy baselines.
  • Tune per category. Start every category at Balanced, tighten the ones that matter most for your audience, and turn a category Off only with a documented exception.

Clean up

A Strict threshold can block legitimate traffic. Remove the model policy before you continue. The next lab sets a gateway-wide baseline instead.

Choose your track
1

Remove the policy

Select Policies. On the Content safety policy's row, select More actions (⋮) › Delete, then Delete again in Delete this policy?. The other policies on the model stay.

Keep the gateway, model, and key from lab 02: the next labs reuse them. azd down and terraform destroy don't touch them, because the Bicep + azd and Terraform tracks created their own gateways.

Next steps

Finished the lab?

Mark it complete to track your progress across paths. It's saved only in this browser.