pyrit.score.SelfAskLikertScorer#
- class SelfAskLikertScorer(*, chat_target: PromptChatTarget, likert_scale_path: str | Path, validator: ScorerPromptValidator | None = None)[source]#
Bases:
FloatScaleScorerA class that represents a “self-ask” score for text scoring for a likert scale.
- __init__(*, chat_target: PromptChatTarget, likert_scale_path: str | Path, validator: ScorerPromptValidator | None = None) None[source]#
Methods
__init__(*, chat_target, likert_scale_path)get_identifier()Returns an identifier dictionary for the scorer.
get_scorer_metrics(dataset_name[, metrics_type])Returns evaluation statistics for the scorer using the dataset_name of the human labeled dataset that this scorer was run against.
scale_value_float(value, min_value, max_value)Scales a value from 0 to 1 based on the given min and max values.
score_async(request_response, *[, ...])Score the request_response, add the results to the database and return a list of Score objects.
score_image_async(image_path, *[, objective])Scores the given image using the chat target.
score_image_batch_async(*, image_paths[, ...])score_prompts_batch_async(*, request_responses)Score multiple prompts in batches using the provided objectives.
score_response_async(*, response[, ...])Score a response using an objective scorer and optional auxiliary scorers.
score_response_multiple_scorers_async(*, ...)Score a response using multiple scorers in parallel.
score_text_async(text, *[, objective])Scores the given text based on the task using the chat target.
set_likert_scale_system_prompt(likert_scale_path)Sets the Likert scale to use for scoring.
validate_return_scores(scores)Validates the scores returned by the scorer.
Attributes
scorer_type