PyRIT
Python Risk Identification Tool¶
Automated and human-led AI red teaming — a flexible, extensible framework for assessing the security and safety of generative AI systems at scale.
What PyRIT Offers
Key Capabilities¶
🎯 Automated Red Teaming
Run multi-turn attack strategies like Crescendo Russinovich et al., 2024, TAP Mehrotra et al., 2023, and Skeleton Key Microsoft Security Response Center, 2024 against AI systems with minimal setup. Single-turn and multi-turn attacks supported out of the box.
📦 Scenario Framework
Run standardized evaluation scenarios at large scale — covering content harms, psychosocial risks, data leakage, and more. Compose strategies and datasets for repeatable, comprehensive assessments across hundreds of objectives.
🖥️ CoPyRIT
A graphical user interface for human-led red teaming. Interact with AI systems directly, track findings, and collaborate with your team — all from a modern web UI.
🔌 Any Target
Test OpenAI, Azure, Anthropic, Google, HuggingFace, custom HTTP endpoints or WebSockets, web app targets with Playwright, or build your own with a simple interface.
💾 Built-in Memory
Track all conversations, scores, and attack results with SQLite or Azure SQL. Export, analyze, and share results with your team.
📊 Flexible Scoring
Evaluate AI responses with true/false, Likert scale, classification, and custom scorers — powered by LLMs, Azure AI Content Safety, or your own logic.
Installation Guide¶
PyRIT offers flexible installation options to suit different needs. Choose the path that best fits your use case.
For Users - Quick Start ⭐
Get started immediately with a pre-configured environment:
✅ All dependencies included
✅ No Python setup needed
✅ JupyterLab built-in
✅ Works on all platforms
For Users - Custom Setup
Install PyRIT directly on your machine:
✅ Full Python environment control
✅ Lighter weight installation
✅ Easy integration with existing workflows
For Contributors ⭐
Standardized development environment:
✅ Pre-configured VS Code setup
✅ Consistent across all contributors
✅ All extensions pre-installed
For Contributors - Custom Dev Setup
Install from source in editable mode:
✅ Full development control
✅ Use any IDE or editor
✅ Customize environment
- Russinovich, M., Salem, A., & Eldan, R. (2024). Great, Now Write an Article About That: The Crescendo Multi-Turn LLM Jailbreak Attack. arXiv Preprint arXiv:2404.01833. https://crescendo-the-multiturn-jailbreak.github.io/
- Mehrotra, A., Zampetakis, M., Kassianik, P., Nelson, B., Anderson, H., Singer, Y., & Karbasi, A. (2023). Tree of Attacks: Jailbreaking Black-Box LLMs Automatically. arXiv Preprint arXiv:2312.02119. https://arxiv.org/abs/2312.02119
- Microsoft Security Response Center. (2024). Mitigating Skeleton Key, a New Type of Generative AI Jailbreak Technique. https://www.microsoft.com/en-us/security/blog/2024/06/26/mitigating-skeleton-key-a-new-type-of-generative-ai-jailbreak-technique/