pyrit.score.create_conversation_scorer

pyrit.score.create_conversation_scorer#

create_conversation_scorer(*, scorer: Scorer, validator: ScorerPromptValidator | None = None) → Scorer[source]#

Create a ConversationScorer that inherits from the same type as the wrapped scorer.

This factory dynamically creates a ConversationScorer class that inherits from the wrapped scorer’s base class (FloatScaleScorer or TrueFalseScorer), ensuring the returned scorer is an instance of both ConversationScorer and the wrapped scorer’s type.

Parameters:

  • scorer (Scorer) – The scorer to wrap for conversation-level evaluation. Must be an instance of FloatScaleScorer or TrueFalseScorer.

  • validator (Optional[ScorerPromptValidator]) – Optional validator override. If not provided, uses the wrapped scorer’s validator.

Returns:

A ConversationScorer instance that is also an instance of the wrapped scorer’s type.

Return type:

Scorer

Raises:

ValueError – If the scorer is not an instance of FloatScaleScorer or TrueFalseScorer.

Example

>>> float_scorer = SelfAskLikertScorer(chat_target=target, likert_scale_path=scale_path)
>>> conversation_scorer = create_conversation_scorer(scorer=float_scorer)
>>> isinstance(conversation_scorer, FloatScaleScorer)  # True
>>> isinstance(conversation_scorer, ConversationScorer)  # True
Contents