Skip to main content
Back to top
Ctrl
+
K
Search
Ctrl
+
K
PyRIT
How to Guide
Install PyRIT
What can I do if Jupyter cannot find PyRIT?
Populating Secrets
Azure SQL Database Setup, Authentication and User Permissions
Contributing
Exception Handling Guidelines
Contribute with Git
Incorporating Research
Installation
Releasing PyRIT to PyPI
Style Guide
Unit Tests, Pre-Commit Hooks, and Notebooks
Architecture
User guide
Orchestrators
1. PromptSendingOrchestrator
Red Teaming Orchestrator
3. Cross-domain Prompt Injection Attacks (XPIA)
4. Scoring Orchestrator
5. Crescendo Orchestrator
6. Skeleton Key Orchestrator
AdvBench PromptSendingOrchestrator - optional
Benchmark Orchestrator - optional
DecodingTrust Stereotypes dataset testing - optional
TDC-23 Red Teaming Dataset - optional
Flip Orchestrator - optional
TrustAIRLab Forbidden Questions Dataset - optional
Fuzzing Jailbreak Templates - optional
HarmBench dataset testing - optional
LLM-LAT/harmful-dataset testing - optional
How to use HITL Scoring - optional
LibrAI “Do Not Answer” Dataset Testing
Many-Shot Jailbreaking - optional
PAIR Orchestrator - optional
PKU-SafeRLHF dataset testing - optional
SecLists Bias Testing - optional
Tree of attacks with pruning - optional
Violent Durian attack strategy - optional
XSTest Bias Testing - optional
Prompt Targets
1. Azure OpenAI Chat Target
2. Creating Custom Targets
3. AML Chat Targets
4. Azure Blob Storage Targets
5. Multi-Modal Targets
6. Rate Limit (RPM) Threshold
7. HTTP Target:
OpenAI Completions - optional
Prompt Shield Target Documentation + Tutorial - optional
HuggingFace Chat Target Testing - optional
Converters
1. Converters with LLMs
2. Using Prompt Converters
3. Audio Converters
4. Image Converters
5. Selectively Converting
6. Human in the Loop Converter
Generating Perturbed Prompts Using the CharSwapGenerator - optional
Jailbreaking Large Language Models with Symbolic Mathematics Using the MathPromptConverter - optional
Scoring
1. Float Scale Scoring using Azure Content Safety API
2. True False Scoring
3. Classification Scoring
4. Float Scale Scoring using Likert Scale
5. Human in the Loop Scoring
6. Refusal Scorer
Insecure Code Scorer
Prompt Shield Scorer Documentation + Tutorial - optional
Batch Scoring with SelfAskTrueFalseScorer
Memory
1. DuckDB Memory
2. Basic Memory Programming Usage
3. PromptRequestPiece and PromptRequestResposne
4. Updating Memory Manually
5. Resending Prompts Using Memory Labels Example
6. Azure SQL Memory
7. PromptSendingOrchestrator with Azure SQL Memory
8. Seed Prompt Database
9. Exporting Data Example
Azure OpenAI Embeddings - optional
Chat messages - optional
Auxiliary Attacks
1. Generating GCG Suffixes Using Azure Machine Learning
Deployments
Deploying Hugging Face Models into Azure ML Managed Online Endpoint
Importing and Registering Hugging Face Models into Azure ML
Hugging Face LLMs on Azure ML: Endpoint Interaction Guide
Score Azure ML Managed Online Endpoint
Troubleshooting Guide for HF Azure ML Models
API Reference
Blog
Repository
Suggest edit
Open issue
.rst
.pdf
pyrit.models.ScoreType
Contents
ScoreType
pyrit.models.ScoreType
#
ScoreType
#
alias of
Literal
[‘true_false’, ‘float_scale’]
Contents
ScoreType
