pyrit.score.SelfAskScaleScorer#
- class SelfAskScaleScorer(*, chat_target: PromptChatTarget, scale_arguments_path: str | Path | None = None, system_prompt_path: str | Path | None = None, validator: ScorerPromptValidator | None = None)[source]#
Bases:
FloatScaleScorer
A class that represents a “self-ask” score for text scoring for a customizable numeric scale.
- __init__(*, chat_target: PromptChatTarget, scale_arguments_path: str | Path | None = None, system_prompt_path: str | Path | None = None, validator: ScorerPromptValidator | None = None) None [source]#
Methods
__init__
(*, chat_target[, ...])get_identifier
()Returns an identifier dictionary for the scorer.
get_scorer_metrics
(dataset_name[, metrics_type])Returns evaluation statistics for the scorer using the dataset_name of the human labeled dataset that this scorer was run against.
scale_value_float
(value, min_value, max_value)Scales a value from 0 to 1 based on the given min and max values.
score_async
(request_response, *[, ...])Score the request_response, add the results to the database and return a list of Score objects.
score_image_async
(image_path, *[, objective])Scores the given image using the chat target.
score_image_batch_async
(*, image_paths[, ...])score_prompts_batch_async
(*, request_responses)Score multiple prompts in batches using the provided objectives.
score_response_async
(*, response[, ...])Score a response using an objective scorer and optional auxiliary scorers.
score_response_multiple_scorers_async
(*, ...)Score a response using multiple scorers in parallel.
score_text_async
(text, *[, objective])Scores the given text based on the task using the chat target.
validate_return_scores
(scores)Validates the scores returned by the scorer.
Attributes
scorer_type
- class ScalePaths(value, names=None, *, module=None, qualname=None, type=None, start=1, boundary=None)[source]#
Bases:
Enum
- CRITERIA_SCALE = PosixPath('/opt/hostedtoolcache/Python/3.11.13/x64/lib/python3.11/site-packages/pyrit/score/config/scales/criteria_example_scale.yaml')#
- TASK_ACHIEVED_SCALE = PosixPath('/opt/hostedtoolcache/Python/3.11.13/x64/lib/python3.11/site-packages/pyrit/score/config/scales/task_achieved_scale.yaml')#
- TREE_OF_ATTACKS_SCALE = PosixPath('/opt/hostedtoolcache/Python/3.11.13/x64/lib/python3.11/site-packages/pyrit/score/config/scales/tree_of_attacks_scale.yaml')#
- class SystemPaths(value, names=None, *, module=None, qualname=None, type=None, start=1, boundary=None)[source]#
Bases:
Enum
- CRITERIA_SYSTEM_PROMPT = PosixPath('/opt/hostedtoolcache/Python/3.11.13/x64/lib/python3.11/site-packages/pyrit/score/config/scales/criteria_system_prompt.yaml')#
- GENERAL_SYSTEM_PROMPT = PosixPath('/opt/hostedtoolcache/Python/3.11.13/x64/lib/python3.11/site-packages/pyrit/score/config/scales/general_system_prompt.yaml')#
- RED_TEAMER_SYSTEM_PROMPT = PosixPath('/opt/hostedtoolcache/Python/3.11.13/x64/lib/python3.11/site-packages/pyrit/score/config/scales/red_teamer_system_prompt.yaml')#