pyrit.score.ObjectiveScorerEvaluator

class ObjectiveScorerEvaluator(scorer: Scorer)

Bases: ScorerEvaluator

A class that evaluates an objective scorer against HumanLabeledDatasets of type OBJECTIVE.

__init__(scorer: Scorer)

Initialize the ScorerEvaluator with a scorer.

Parameters:

scorer (Scorer) – The scorer to evaluate.

Methods

__init__(scorer)	Initialize the ScorerEvaluator with a scorer.
from_scorer(scorer[, metrics_type])	Create a ScorerEvaluator based on the type of scoring.
run_evaluation_async(*, dataset_files[, ...])	Evaluate scorer using dataset files configuration.

Attributes

expected_metrics_type

expected_metrics_type: MetricsType = 'objective'