pyrit.models.ScenarioResult

class ScenarioResult(*, scenario_identifier: ScenarioIdentifier, objective_target_identifier: dict[str, str], attack_results: dict[str, List[AttackResult]], objective_scorer: Scorer | None = None, objective_scorer_identifier: dict[str, str] | None = None, scenario_run_state: Literal['CREATED', 'IN_PROGRESS', 'COMPLETED', 'FAILED'] = 'CREATED', labels: dict[str, str] | None = None, completion_time: datetime | None = None, number_tries: int = 0, id: UUID | None = None)

Bases: object

Scenario result class for aggregating scenario results.

Note: When creating a new ScenarioResult, objective_scorer should always be provided. The parameter is Optional only to support deserialization from the database where the scorer object cannot be reconstructed.

__init__(*, scenario_identifier: ScenarioIdentifier, objective_target_identifier: dict[str, str], attack_results: dict[str, List[AttackResult]], objective_scorer: Scorer | None = None, objective_scorer_identifier: dict[str, str] | None = None, scenario_run_state: Literal['CREATED', 'IN_PROGRESS', 'COMPLETED', 'FAILED'] = 'CREATED', labels: dict[str, str] | None = None, completion_time: datetime | None = None, number_tries: int = 0, id: UUID | None = None) → None

Methods

__init__(*, scenario_identifier, ...[, ...])
get_objective_scorer_identifier()	Get the objective scorer identifier as a ScorerIdentifier object.
get_objectives(*[, atomic_attack_name])	Get the list of unique objectives for this scenario.
get_scorer_evaluation_metrics()	Get the evaluation metrics for the scenario's scorer from the scorer evaluation registry.
get_strategies_used()	Get the list of strategies used in this scenario.
normalize_scenario_name(scenario_name)	Normalize a scenario name to match the stored class name format.
objective_achieved_rate(*[, atomic_attack_name])	Get the success rate of this scenario.

Attributes

objective_scorer

Get the objective scorer for this scenario.

get_objective_scorer_identifier() → ScorerIdentifier | None

Get the objective scorer identifier as a ScorerIdentifier object.

Reconstructs a ScorerIdentifier from the stored dict representation. This can be used even for deserialized results from the database.

Returns:

The scorer identifier object, or None if no identifier is stored.

Return type:

ScorerIdentifier

get_objectives(*, atomic_attack_name: str | None = None) → List[str]

Get the list of unique objectives for this scenario.

Parameters:

atomic_attack_name (Optional[str]) – Name of specific atomic attack to include. If None, includes objectives from all atomic attacks. Defaults to None.

Returns:

Deduplicated list of objectives.

Return type:

List[str]

get_scorer_evaluation_metrics() → ScorerMetrics | None

Get the evaluation metrics for the scenario’s scorer from the scorer evaluation registry.

Returns:

The evaluation metrics object, or None if not found.

Return type:

ScorerMetrics

get_strategies_used() → List[str]

Get the list of strategies used in this scenario.

static normalize_scenario_name(scenario_name: str) → str

Normalize a scenario name to match the stored class name format.

Converts CLI-style snake_case names (e.g., “foundry” or “content_harms”) to PascalCase class names (e.g., “Foundry” or “ContentHarms”) for database queries. If the input is already in PascalCase or doesn’t match the snake_case pattern, it is returned unchanged.

This is the inverse of ScenarioRegistry._class_name_to_scenario_name().

Parameters:

scenario_name – The scenario name to normalize.

Returns:

The normalized scenario name suitable for database queries.

objective_achieved_rate(*, atomic_attack_name: str | None = None) → int

Get the success rate of this scenario.

Parameters:

atomic_attack_name (Optional[str]) – Name of specific atomic attack to calculate rate for. If None, calculates rate across all atomic attacks. Defaults to None.

Returns:

Success rate as a percentage (0-100).

Return type:

int

property objective_scorer: Scorer | None

Get the objective scorer for this scenario.

Returns:

The objective scorer instance, or None if deserialized from database.

Return type:

Scorer