pyrit.score.HarmScorerEvaluator

class HarmScorerEvaluator(scorer: Scorer)

Bases: ScorerEvaluator

A class that evaluates a harm scorer against HumanLabeledDatasets of type HARM.

__init__(scorer: Scorer)

Initialize the ScorerEvaluator with a scorer.

Parameters:

scorer (Scorer) – The scorer to evaluate.

Methods

__init__(scorer)	Initialize the ScorerEvaluator with a scorer.
from_scorer(scorer[, metrics_type])	Create a ScorerEvaluator based on the type of scoring.
run_evaluation_async(*, dataset_files[, ...])	Evaluate scorer using dataset files configuration.

Attributes

expected_metrics_type

expected_metrics_type: MetricsType = 'harm'