pyrit.analytics

Analytics module for PyRIT conversation and result analysis.

Functions¶

analyze_results¶

analyze_results(attack_results: list[AttackResult]) → dict[str, AttackStats | dict[str, AttackStats]]

Analyze a list of AttackResult objects and return overall and grouped statistics.

Returns:

• dict[str, AttackStats | dict[str, AttackStats]] — A dictionary of AttackStats objects. The overall stats are accessible with the key

• dict[str, AttackStats | dict[str, AttackStats]] — “Overall”, and the stats of any attack can be retrieved using “By_attack_identifier”

• dict[str, AttackStats | dict[str, AttackStats]] — followed by the identifier of the attack.

Raises:

• ValueError — if attack_results is empty.

• TypeError — if any element is not an AttackResult.

ApproximateTextMatching¶

Bases: TextMatching

Approximate text matching using n-gram overlap.

This strategy computes the proportion of character n-grams from the target that are present in the text. Useful for detecting partial matches, encoded content, or text with variations.

Constructor Parameters:

Methods:

get_overlap_score¶

get_overlap_score(target: str, text: str) → float

Get the n-gram overlap score without threshold comparison.

Useful for getting detailed scoring information.

Returns:

• float — The n-gram overlap score between 0.0 and 1.0.

is_match¶

is_match(target: str, text: str) → bool

Check if target approximately matches text using n-gram overlap.

Returns:

• bool — True if n-gram overlap score exceeds threshold, False otherwise.

AttackStats¶

Statistics for attack analysis results.

ConversationAnalytics¶

Handles analytics operations on conversation data, such as finding similar chat messages based on conversation history or embedding similarity.

Constructor Parameters:

Methods:

get_prompt_entries_with_same_converted_content¶

get_prompt_entries_with_same_converted_content(chat_message_content: str) → list[ConversationMessageWithSimilarity]

Retrieve chat messages that have the same converted content.

Returns:

• list[ConversationMessageWithSimilarity] — list[ConversationMessageWithSimilarity]: A list of ConversationMessageWithSimilarity objects representing

• list[ConversationMessageWithSimilarity] — the similar chat messages based on content.

get_similar_chat_messages_by_embedding¶

get_similar_chat_messages_by_embedding(chat_message_embedding: list[float], threshold: float = 0.8) → list[EmbeddingMessageWithSimilarity]

Retrieve chat messages that are similar to the given embedding based on cosine similarity.

Returns:

• list[EmbeddingMessageWithSimilarity] — List[ConversationMessageWithSimilarity]: A list of ConversationMessageWithSimilarity objects representing

• list[EmbeddingMessageWithSimilarity] — the similar chat messages based on embedding similarity.

ExactTextMatching¶

Bases: TextMatching

Exact substring matching strategy.

Checks if the target string is present in the text as a substring.

Constructor Parameters:

Methods:

is_match¶

is_match(target: str, text: str) → bool

Check if target string is present in text.

Returns:

• bool — True if target is found in text, False otherwise.

TextMatching¶

Bases: Protocol

Protocol for text matching strategies.

Classes implementing this protocol must provide an is_match method that checks if a target string matches text according to some strategy.

Methods:

is_match¶

is_match(target: str, text: str) → bool

Check if target matches text according to the strategy.

Returns:

• bool — True if target matches text according to the strategy, False otherwise.