pyrit.score.ScorerPromptValidator

pyrit.score.ScorerPromptValidator#

class ScorerPromptValidator(*, supported_data_types: Sequence[Literal['text', 'image_path', 'audio_path', 'video_path', 'url', 'reasoning', 'error', 'function_call', 'tool_call', 'function_call_output']] | None = None, required_metadata: Sequence[str] | None = None, supported_roles: Sequence[Literal['system', 'user', 'assistant', 'tool', 'developer']] | None = None, max_pieces_in_response: int | None = None, max_text_length: int | None = None, enforce_all_pieces_valid: bool | None = False, raise_on_no_valid_pieces: bool | None = True, is_objective_required=False)[source]#

Bases: object

Validates message pieces and scorer configurations.

This class provides validation for scorer inputs, ensuring that message pieces meet required criteria such as data types, roles, and metadata requirements.

__init__(*, supported_data_types: Sequence[Literal['text', 'image_path', 'audio_path', 'video_path', 'url', 'reasoning', 'error', 'function_call', 'tool_call', 'function_call_output']] | None = None, required_metadata: Sequence[str] | None = None, supported_roles: Sequence[Literal['system', 'user', 'assistant', 'tool', 'developer']] | None = None, max_pieces_in_response: int | None = None, max_text_length: int | None = None, enforce_all_pieces_valid: bool | None = False, raise_on_no_valid_pieces: bool | None = True, is_objective_required=False)[source]#

Initialize the ScorerPromptValidator.

Parameters:

  • supported_data_types (Optional[Sequence[PromptDataType]]) – Data types that the scorer supports. Defaults to all data types if not provided.

  • required_metadata (Optional[Sequence[str]]) – Metadata keys that must be present in message pieces. Defaults to empty list.

  • supported_roles (Optional[Sequence[ChatMessageRole]]) – Message roles that the scorer supports. Defaults to all roles if not provided.

  • max_pieces_in_response (Optional[int]) – Maximum number of pieces allowed in a response. Defaults to None (no limit).

  • max_text_length (Optional[int]) – Maximum character length for text data type pieces. Defaults to None (no limit).

  • enforce_all_pieces_valid (Optional[bool]) – Whether all pieces must be valid or just at least one. Defaults to False.

  • raise_on_no_valid_pieces (Optional[bool]) – Whether to raise ValueError when no pieces are valid. Defaults to True for backwards compatibility. Set to False to allow empty scores.

  • is_objective_required (bool) – Whether an objective must be provided for scoring. Defaults to False.

Methods

__init__(*[, supported_data_types, ...])

Initialize the ScorerPromptValidator.

is_message_piece_supported(message_piece)

Check if a message piece is supported by this validator.

validate(message, objective)

Validate a message and objective against configured requirements.
is_message_piece_supported(message_piece: MessagePiece) → bool[source]#

Check if a message piece is supported by this validator.

Parameters:

message_piece (MessagePiece) – The message piece to check.

Returns:

True if the message piece meets all validation criteria, False otherwise.

Return type:

bool

validate(message: Message, objective: str | None) → None[source]#

Validate a message and objective against configured requirements.

Parameters:

  • message (Message) – The message to validate.

  • objective (str | None) – The objective string, if required.

Raises:

ValueError – If validation fails due to unsupported pieces, exceeding max pieces, or missing objective.

Contents