Ship¶

This page explains how AgentOps gates a release in CI/CD. AgentOps owns the repo-side quality gate; your platform owns infrastructure and deployment. The goal is simple: the agent version that gets deployed is the exact version that was evaluated.

For the full GitHub Actions reference, including every workflow file, the YAML, and the OIDC and RBAC setup steps, see AgentOps on GitHub Actions. This page is the overview that explains why the pieces fit together.

Branches and environments¶

AgentOps assumes a GitFlow-style branch model. Feature PRs run an eval against the candidate agent before merge. The PR from release/** to main is a manual approval gate for the already-tested release branch; it does not call agents. Deploy workflows run after merge and promote the reviewed branch to its environment:

flowchart LR
    feature["feature/*"] --> prDev["PR eval<br/>candidate"]
    prDev --> sandbox["sandbox"]
    prDev --> develop["develop"]
    develop --> devDeploy["Eval + deploy<br/>agentops-deploy-dev"]
    devDeploy --> devEnv["dev"]

    develop --> release["release/*"]
    release --> qaDeploy["Eval + deploy<br/>agentops-deploy-qa"]
    qaDeploy --> qaEnv["qa"]

    release --> prProd["PR: release to main<br/>manual approval"]
    prProd --> main["main"]
    main --> prodDeploy["Prod release process<br/>deploy + smoke test<br/>agentops-deploy-prod"]
    prodDeploy --> prodEnv["production"]

    classDef branch fill:#e7f0fd,stroke:#1f4e79,color:#000;
    classDef pipeline fill:#ede7f6,stroke:#4527a0,color:#000;
    classDef env fill:#d1ecf1,stroke:#0c5460,color:#000;
    class feature,develop,release,main branch;
    class prDev,devDeploy,qaDeploy,prProd,prodDeploy pipeline;
    class sandbox,devEnv,qaEnv,prodEnv env;

Legend: Git branch PR or workflow gate deployed environment

The PR gate and the deploy workflows each have one job, so it helps to read the flow as two tables: what runs on a pull request, and what runs after a merge.

Pull request	What runs	Notes
Feature PR into `develop` or `release/**`	`agentops-pr.yml` evaluates the PR candidate.	Protects the change before it enters the branch. It does not validate the already-deployed dev app. For HTTP agents it evaluates the sandbox endpoint; for prompt agents it stages and evaluates the candidate prompt in sandbox.
PR from `release/**` into `main`	Manual approval gate with static checks only.	No agents are called; it approves the already-tested release branch.

Merge into	Deploy workflow	Target environment
`develop`	per-environment deploy	dev
`release/**`	per-environment deploy	QA
`main`	`agentops-deploy-prod` production release process (deploy and smoke test)	production

Where the full setup lives

The two workflows you start with are covered next. The full set, including the workflow YAML and the GitHub Environment and OIDC setup, is in AgentOps on GitHub Actions.

The two core workflows¶

A generated AgentOps scaffold ships a PR gate and per-environment deploy workflows. The two that matter most early are the PR gate and the dev deploy.

Workflow	Trigger	What it does
`agentops-pr.yml`	PRs to `develop`, `release/**`	Evaluates the PR candidate, runs the Doctor gate, and comments on the PR.
`agentops-deploy-dev.yml`	push to `develop`	Evaluates, then builds and deploys to the dev environment.

You do not write these by hand. agentops workflow analyze reads the repo and recommends the deploy wiring and eval runner, and agentops workflow generate writes the workflow files from that recommendation. Because both use the same analysis, the plan and the generated files do not drift.

Generate the PR gate first

Start with agentops workflow generate --kinds pr. Add the dev, qa, and prod deploys only after GitHub Environments and Azure OIDC are ready. This keeps your first green run small and avoids wiring deploy steps before the gate works.

Candidate versioning¶

For Foundry prompt agents, each deploy stages a candidate version from your source-controlled prompt_file before evaluating it. PR-run candidates are tagged in Foundry with agentops:candidate=true plus the PR number and a timestamp, so portal viewers can tell abandoned PR candidates apart from deployed versions of record.

The deployed-of-record version is tracked in foundry-agent.json, written per environment as a workflow artifact after the gate passes. That file, not the Foundry version number, is the supported way to know what each environment runs.

Prompt SHA and git SHA are the durable identity

Foundry version numbers are local to each project, so sandbox travel-agent:2 may not match the dev or prod number. AgentOps instead records agentops.prompt_sha256 (the prompt text) and agentops.git_sha (the commit) on every version and in foundry-agent.json. To check whether two environments run the same prompt, compare those SHAs, not the version numbers.

This is the invariant the whole flow protects: the evaluated agent version is the deployed agent version. Foundry manages the candidate versions; AgentOps supplies the gate, the deployment record, and Cockpit visibility.

Why use a candidate¶

The PR gate validates the proposed agent before it enters develop or a release/** branch. For HTTP agents, that usually means the sandbox endpoint you configured in the workflow. For Foundry prompt agents, the workflow stages a throwaway prompt version in sandbox and evaluates that candidate. The dev project is updated only by the deploy workflow after the PR merges.

A passing PR gate is evidence that the proposed change is safe to merge. The dev deploy then promotes the reviewed branch and records the deployed prompt SHA.

The Doctor gate in CI¶

Every PR run also runs agentops doctor --evidence-pack after the eval step. The eval step is the hard merge gate; the Doctor gate adds readiness checks like regression detection against the rolling baseline.

`--doctor-gate` value	PR behavior
`critical` (default)	Blocks the PR on critical findings, including a regression that drops a metric well below baseline even when eval thresholds still pass.
`warning`	Also blocks on smaller regression drops.
`none`	Doctor still writes and uploads evidence, but does not block; the eval step stays the only hard gate.

Production deploy templates always run Doctor with a critical finding gate. To understand the findings and severities behind these gates, see Operate and the Doctor checks reference.

Identity and access¶

CI authenticates to Azure with GitHub OIDC and federated credentials, so no long-lived secrets live in the repo. The same principal needs the right Azure RBAC roles before the first run, or the eval step fails with null metrics.

Two roles are required for prompt-agent gates

The OIDC principal needs Foundry User on the Foundry project and Cognitive Services OpenAI User on the AI Services account that hosts the judge model. If only one is in place, every metric returns null and the gate fails without an obvious cause. The full setup, including the role ids and the federation steps, is in AgentOps on GitHub Actions.

For the underlying procedures, follow Microsoft's own docs rather than copying long steps here:

Try it¶

Generate the CI/CD workflows from the same analysis AgentOps uses, smallest gate first.

Read the repo and recommend the deploy wiring and eval runner.
```
agentops workflow analyze
```
Generate just the PR gate first so your first green run stays small.
```
agentops workflow generate --kinds pr
```
Add the per-environment deploy workflows once GitHub Environments and Azure OIDC are ready.
```
agentops workflow generate --kinds pr,dev,qa,prod
```
Target Azure DevOps Pipelines instead of GitHub Actions when that is your platform.
```
agentops workflow generate --kinds pr --platform azure-devops
```

Run from your coding agent¶

Install the AgentOps skills so your coding agent can wire and explain the pipeline for you.

agentops skills install --platform copilot

The skill that maps to shipping is:

Skill	What it helps with
`agentops-workflow`	Set up, generate, and explain CI/CD workflows.

Next¶

Read the full GitHub Actions reference, score readiness after each run on the Operate page, or see where production signal comes from on the Observe page.