pyrit.score.ScorerPromptValidator

pyrit.score.ScorerPromptValidator#

class ScorerPromptValidator(*, supported_data_types: Sequence[Literal['text', 'image_path', 'audio_path', 'video_path', 'binary_path', 'url', 'reasoning', 'error', 'function_call', 'tool_call', 'function_call_output']] | None = None, required_metadata: Sequence[str] | None = None, supported_roles: Sequence[Literal['system', 'user', 'assistant', 'simulated_assistant', 'tool', 'developer']] | None = None, max_pieces_in_response: int | None = None, max_text_length: int | None = None, enforce_all_pieces_valid: bool | None = False, raise_on_no_valid_pieces: bool | None = False, is_objective_required: bool = False)[source]#

Bases: object

Validates message pieces and scorer configurations.

This class provides validation for scorer inputs, ensuring that message pieces meet required criteria such as data types, roles, and metadata requirements.

__init__(*, supported_data_types: Sequence[Literal['text', 'image_path', 'audio_path', 'video_path', 'binary_path', 'url', 'reasoning', 'error', 'function_call', 'tool_call', 'function_call_output']] | None = None, required_metadata: Sequence[str] | None = None, supported_roles: Sequence[Literal['system', 'user', 'assistant', 'simulated_assistant', 'tool', 'developer']] | None = None, max_pieces_in_response: int | None = None, max_text_length: int | None = None, enforce_all_pieces_valid: bool | None = False, raise_on_no_valid_pieces: bool | None = False, is_objective_required: bool = False) → None[source]#

Initialize the ScorerPromptValidator.

Parameters:

  • supported_data_types (Optional[Sequence[PromptDataType]]) – Data types that the scorer supports. Defaults to all data types if not provided.

  • required_metadata (Optional[Sequence[str]]) – Metadata keys that must be present in message pieces. Defaults to empty list.

  • supported_roles (Optional[Sequence[ChatMessageRole]]) – Message roles that the scorer supports. Defaults to all roles if not provided.

  • max_pieces_in_response (Optional[int]) – Maximum number of pieces allowed in a response. Defaults to None (no limit).

  • max_text_length (Optional[int]) – Maximum character length for text data type pieces. Defaults to None (no limit).

  • enforce_all_pieces_valid (Optional[bool]) – Whether all pieces must be valid or just at least one. Defaults to False.

  • raise_on_no_valid_pieces (Optional[bool]) – Whether to raise ValueError when no pieces are valid. Defaults to False, allowing scorers to handle empty results gracefully (e.g., returning False for blocked responses). Set to True to raise an exception instead.

  • is_objective_required (bool) – Whether an objective must be provided for scoring. Defaults to False.

Methods

__init__(*[, supported_data_types, ...])

Initialize the ScorerPromptValidator.

is_message_piece_supported(message_piece)

Check if a message piece is supported by this validator.

validate(message, objective)

Validate a message and objective against configured requirements.
is_message_piece_supported(message_piece: MessagePiece) → bool[source]#

Check if a message piece is supported by this validator.

Parameters:

message_piece (MessagePiece) – The message piece to check.

Returns:

True if the message piece meets all validation criteria, False otherwise.

Return type:

bool

validate(message: Message, objective: str | None) → None[source]#

Validate a message and objective against configured requirements.

Parameters:

  • message (Message) – The message to validate.

  • objective (str | None) – The objective string, if required.

Raises:

ValueError – If validation fails due to unsupported pieces, exceeding max pieces, or missing objective.

Contents