pyrit.score.CompositeScorer

pyrit.score.CompositeScorer#

class CompositeScorer(*, aggregator: Callable[[Iterable[Score]], ScoreAggregatorResult], scorers: List[Scorer], score_category: str | None = None)[source]#

Bases: Scorer

A scorer that aggregates other true_false scorers using a specified aggregation function.

It returns a single score of True or False based on the aggregation of the scores of the constituent scorers.

Parameters:

  • aggregator – The aggregation function to use (e.g. AND_, OR_, MAJORITY_)

  • scorers – List of true_false scorers to combine

  • score_category – Optional category for the score

__init__(*, aggregator: Callable[[Iterable[Score]], ScoreAggregatorResult], scorers: List[Scorer], score_category: str | None = None) → None[source]#

Methods

__init__(*, aggregator, scorers[, ...])

get_identifier()

Returns an identifier dictionary for the scorer.

scale_value_float(value, min_value, max_value)

Scales a value from 0 to 1 based on the given min and max values.

score_async(request_response, *[, task])

Scores the request response by combining results from all constituent scorers.

score_image_async(image_path, *[, task])

Scores the given image using the chat target.

score_prompts_with_tasks_batch_async(*, ...)

score_responses_inferring_tasks_batch_async(*, ...)

Scores a batch of responses (ignores non-assistant messages).

score_text_async(text, *[, task])

Scores the given text based on the task using the chat target.

validate(request_response, *[, task])

Validates the request response for scoring.

Attributes

scorer_type

async score_async(request_response: PromptRequestPiece, *, task: str | None = None) → List[Score][source]#

Scores the request response by combining results from all constituent scorers.

Parameters:

  • request_response – The request response to be scored

  • task – Optional task description for scoring context

Returns:

List containing a single Score object representing the combined result

scorer_type: Literal['true_false', 'float_scale']#
validate(request_response: PromptRequestPiece, *, task: str | None = None) → None[source]#

Validates the request response for scoring.

Parameters:

  • request_response – The request response to validate

  • task – Optional task description for validation context

Contents