pyrit.score.LookBackScorer#
- class LookBackScorer(chat_target: PromptChatTarget, *, exclude_instruction_prompts: bool, validator: ScorerPromptValidator | None = None)[source]#
Bases:
FloatScaleScorer
Create a score from analyzing the entire conversation and adds them to the database.
- Parameters:
chat_target (PromptChatTarget) – The chat target to use for scoring.
exclude_instruction_prompts (bool) – If True, excludes instruction prompts from the conversation. Must be passed as a keyword argument.
- __init__(chat_target: PromptChatTarget, *, exclude_instruction_prompts: bool, validator: ScorerPromptValidator | None = None) None [source]#
Methods
__init__
(chat_target, *, ...[, validator])get_identifier
()Returns an identifier dictionary for the scorer.
get_scorer_metrics
(dataset_name[, metrics_type])Returns evaluation statistics for the scorer using the dataset_name of the human labeled dataset that this scorer was run against.
scale_value_float
(value, min_value, max_value)Scales a value from 0 to 1 based on the given min and max values.
score_async
(request_response, *[, ...])Score the request_response, add the results to the database and return a list of Score objects.
score_image_async
(image_path, *[, objective])Scores the given image using the chat target.
score_image_batch_async
(*, image_paths[, ...])score_prompts_batch_async
(*, request_responses)Score multiple prompts in batches using the provided objectives.
score_response_async
(*, response[, ...])Score a response using an objective scorer and optional auxiliary scorers.
score_response_multiple_scorers_async
(*, ...)Score a response using multiple scorers in parallel.
score_text_async
(text, *[, objective])Scores the given text based on the task using the chat target.
validate_return_scores
(scores)Validates the scores returned by the scorer.
Attributes
scorer_type