pyrit.score.PromptShieldScorer#
- class PromptShieldScorer(*, prompt_shield_target: ~pyrit.prompt_target.prompt_shield_target.PromptShieldTarget, validator: ~pyrit.score.scorer_prompt_validator.ScorerPromptValidator | None = None, score_aggregator: ~typing.Callable[[~typing.Iterable[~pyrit.models.score.Score]], ~pyrit.score.score_aggregator_result.ScoreAggregatorResult] = <function _create_aggregator.<locals>.aggregator>)[source]#
Bases:
TrueFalseScorer
Returns true if an attack or jailbreak has been detected by Prompt Shield.
- __init__(*, prompt_shield_target: ~pyrit.prompt_target.prompt_shield_target.PromptShieldTarget, validator: ~pyrit.score.scorer_prompt_validator.ScorerPromptValidator | None = None, score_aggregator: ~typing.Callable[[~typing.Iterable[~pyrit.models.score.Score]], ~pyrit.score.score_aggregator_result.ScoreAggregatorResult] = <function _create_aggregator.<locals>.aggregator>) None [source]#
Initialize the PromptShieldScorer.
- Parameters:
prompt_shield_target (PromptShieldTarget) – The Prompt Shield target to use for scoring.
validator (Optional[ScorerPromptValidator]) – Custom validator. Defaults to None.
score_aggregator (TrueFalseAggregatorFunc) – The aggregator function to use. Defaults to TrueFalseScoreAggregator.OR.
Methods
__init__
(*, prompt_shield_target[, ...])Initialize the PromptShieldScorer.
get_identifier
()Returns an identifier dictionary for the scorer.
get_scorer_metrics
(dataset_name[, metrics_type])Returns evaluation statistics for the scorer using the dataset_name of the human labeled dataset that this scorer was run against.
scale_value_float
(value, min_value, max_value)Scales a value from 0 to 1 based on the given min and max values.
score_async
(request_response, *[, ...])Score the request_response, add the results to the database and return a list of Score objects.
score_image_async
(image_path, *[, objective])Scores the given image using the chat target.
score_image_batch_async
(*, image_paths[, ...])score_prompts_batch_async
(*, request_responses)Score multiple prompts in batches using the provided objectives.
score_response_async
(*, response[, ...])Score a response using an objective scorer and optional auxiliary scorers.
score_response_multiple_scorers_async
(*, ...)Score a response using multiple scorers in parallel.
score_text_async
(text, *[, objective])Scores the given text based on the task using the chat target.
validate_return_scores
(scores)Validates the scores returned by the scorer.
Attributes