pyrit.models.ScenarioResult

class ScenarioResult(*, scenario_identifier: ScenarioIdentifier, objective_target_identifier: Dict[str, Any] | TargetIdentifier, attack_results: dict[str, List[AttackResult]], objective_scorer_identifier: Dict[str, Any] | ScorerIdentifier, scenario_run_state: Literal['CREATED', 'IN_PROGRESS', 'COMPLETED', 'FAILED'] = 'CREATED', labels: dict[str, str] | None = None, completion_time: datetime | None = None, number_tries: int = 0, id: UUID | None = None, objective_scorer: Scorer | None = None)

Bases: object

Scenario result class for aggregating scenario results.

__init__(*, scenario_identifier: ScenarioIdentifier, objective_target_identifier: Dict[str, Any] | TargetIdentifier, attack_results: dict[str, List[AttackResult]], objective_scorer_identifier: Dict[str, Any] | ScorerIdentifier, scenario_run_state: Literal['CREATED', 'IN_PROGRESS', 'COMPLETED', 'FAILED'] = 'CREATED', labels: dict[str, str] | None = None, completion_time: datetime | None = None, number_tries: int = 0, id: UUID | None = None, objective_scorer: Scorer | None = None) → None

Methods

__init__(*, scenario_identifier, ...[, ...])
get_objectives(*[, atomic_attack_name])	Get the list of unique objectives for this scenario.
get_scorer_evaluation_metrics()	Get the evaluation metrics for the scenario's scorer from the scorer evaluation registry.
get_strategies_used()	Get the list of strategies used in this scenario.
normalize_scenario_name(scenario_name)	Normalize a scenario name to match the stored class name format.
objective_achieved_rate(*[, atomic_attack_name])	Get the success rate of this scenario.

get_objectives(*, atomic_attack_name: str | None = None) → List[str]

Get the list of unique objectives for this scenario.

Parameters:

atomic_attack_name (Optional[str]) – Name of specific atomic attack to include. If None, includes objectives from all atomic attacks. Defaults to None.

Returns:

Deduplicated list of objectives.

Return type:

List[str]

get_scorer_evaluation_metrics() → ScorerMetrics | None

Get the evaluation metrics for the scenario’s scorer from the scorer evaluation registry.

Returns:

The evaluation metrics object, or None if not found.

Return type:

ScorerMetrics

get_strategies_used() → List[str]

Get the list of strategies used in this scenario.

static normalize_scenario_name(scenario_name: str) → str

Normalize a scenario name to match the stored class name format.

Converts CLI-style snake_case names (e.g., “foundry” or “content_harms”) to PascalCase class names (e.g., “Foundry” or “ContentHarms”) for database queries. If the input is already in PascalCase or doesn’t match the snake_case pattern, it is returned unchanged.

This is the inverse of ScenarioRegistry._class_name_to_scenario_name().

Parameters:

scenario_name – The scenario name to normalize.

Returns:

The normalized scenario name suitable for database queries.

objective_achieved_rate(*, atomic_attack_name: str | None = None) → int

Get the success rate of this scenario.

Parameters:

atomic_attack_name (Optional[str]) – Name of specific atomic attack to calculate rate for. If None, calculates rate across all atomic attacks. Defaults to None.

Returns:

Success rate as a percentage (0-100).

Return type:

int