10. Exception Handling Guidelines

10. Exception Handling Guidelines#

In our PyRIT framework, proper exception handling is crucial for maintaining robustness and reliability. We have centralized exceptions in a dedicated module to streamline this process. When working with attacks, targets, converters, and scorers, and handle exceptions please adhere to the following guidelines:

General Guidelines#

  1. Centralized Exceptions: Use the exceptions defined in our centralized exceptions module or create new ones as necessary in the same module.

  2. Inherit PyritException: Ensure any new exceptions inherit from PyritException to maintain consistency.

  3. Exception Processing: Utilize the process_exception method to handle exceptions appropriately.

  4. Add Response Entries to Memory: After handling an exception using the process_exception method. While adding response entries to memory, ensure to set the response_type and error parameter to error to help identify the responses in the database for easy filtering. The original_value and converted_value or the MessagePiece object should contain the error message.

Specific Scenarios#

  1. Response is None, JSON Parsing Failure in Scorer, RateLimitError

    • Action: In the endpoint code (scoring, target, or converter) there should be a retry mechanism to attempt the operation a few times. For example, use the @pyrit_target_retry decorator

    • If Still Failing:

      • The endpoint raises the exception (it should be unhandled)

      • The top-level attacks can handle the exception as needed

  2. BadRequestError Caused by Content Filter

    • Action: Do not retry the operation.

    • If Occurring: At the endpoint code (target) use the process_exception method to handle the failure and log the error. Do not re-raise.

  3. Unknown Exception

    • Action: Raise the exception itself to allow for proper propagation. Do not retry.

    • Future Learning: Monitor these exceptions to learn and identify patterns for future enhancements and more specific exception handling.

By following these guidelines, we ensure a consistent and robust approach to exception handling across the framework.

Contents