pyrit.score.LookBackScorer

pyrit.score.LookBackScorer#

class LookBackScorer(chat_target: PromptChatTarget, *, exclude_instruction_prompts: bool, validator: ScorerPromptValidator | None = None)[source]#

Create a score from analyzing the entire conversation and adds them to the database.

Parameters:

chat_target (PromptChatTarget) – The chat target to use for scoring.
exclude_instruction_prompts (bool) – If True, excludes instruction prompts from the conversation. Must be passed as a keyword argument.

__init__(chat_target: PromptChatTarget, *, exclude_instruction_prompts: bool, validator: ScorerPromptValidator | None = None) → None[source]#

Methods

`__init__`(chat_target, *, ...[, validator])
`get_identifier`()	Returns an identifier dictionary for the scorer.
`get_scorer_metrics`(dataset_name[, metrics_type])	Returns evaluation statistics for the scorer using the dataset_name of the human labeled dataset that this scorer was run against.
`scale_value_float`(value, min_value, max_value)	Scales a value from 0 to 1 based on the given min and max values.
`score_async`(message, *[, objective, ...])	Score the message, add the results to the database and return a list of Score objects.
`score_image_async`(image_path, *[, objective])	Scores the given image using the chat target.
`score_image_batch_async`(*, image_paths[, ...])
`score_prompts_batch_async`(*, messages[, ...])	Score multiple prompts in batches using the provided objectives.
`score_response_async`(*, response[, ...])	Score a response using an objective scorer and optional auxiliary scorers.
`score_response_multiple_scorers_async`(*, ...)	Score a response using multiple scorers in parallel.
`score_text_async`(text, *[, objective])	Scores the given text based on the task using the chat target.
`validate_return_scores`(scores)	Validates the scores returned by the scorer.

Attributes

scorer_type