pyrit.prompt_target.AzureMLChatTarget#
- class AzureMLChatTarget(*, endpoint: str = None, api_key: str = None, chat_message_normalizer: ~pyrit.chat_message_normalizer.chat_message_normalizer.ChatMessageNormalizer = <pyrit.chat_message_normalizer.chat_message_nop.ChatMessageNop object>, max_new_tokens: int = 400, temperature: float = 1.0, top_p: float = 1.0, repetition_penalty: float = 1.0, max_requests_per_minute: int | None = None, **param_kwargs)[source]#
Bases:
PromptChatTarget- __init__(*, endpoint: str = None, api_key: str = None, chat_message_normalizer: ~pyrit.chat_message_normalizer.chat_message_normalizer.ChatMessageNormalizer = <pyrit.chat_message_normalizer.chat_message_nop.ChatMessageNop object>, max_new_tokens: int = 400, temperature: float = 1.0, top_p: float = 1.0, repetition_penalty: float = 1.0, max_requests_per_minute: int | None = None, **param_kwargs) None[source]#
Initializes an instance of the AzureMLChatTarget class. This class works with most chat completion Instruct models deployed on Azure AI Machine Learning Studio endpoints (including but not limited to: mistralai-Mixtral-8x7B-Instruct-v01, mistralai-Mistral-7B-Instruct-v01, Phi-3.5-MoE-instruct, Phi-3-mini-4k-instruct, Llama-3.2-3B-Instruct, and Meta-Llama-3.1-8B-Instruct). Please create or adjust environment variables (endpoint and key) as needed for the model you are using.
- Parameters:
endpoint (str, Optional) – The endpoint URL for the deployed Azure ML model. Defaults to the value of the AZURE_ML_MANAGED_ENDPOINT environment variable.
api_key (str, Optional) – The API key for accessing the Azure ML endpoint. Defaults to the value of the AZURE_ML_KEY environment variable.
chat_message_normalizer (ChatMessageNormalizer, Optional) – The chat message normalizer. For models that do not allow system prompts such as mistralai-Mixtral-8x7B-Instruct-v01, GenericSystemSquash() can be passed in. Defaults to ChatMessageNop(), which does not alter the chat messages.
max_new_tokens (int, Optional) – The maximum number of tokens to generate in the response. Defaults to 400.
temperature (float, Optional) – The temperature for generating diverse responses. 1.0 is most random, 0.0 is least random. Defaults to 1.0.
top_p (float, Optional) – The top-p value for generating diverse responses. It represents the cumulative probability of the top tokens to keep. Defaults to 1.0.
repetition_penalty (float, Optional) – The repetition penalty for generating diverse responses. 1.0 means no penalty with a greater value (up to 2.0) meaning more penalty for repeating tokens. Defaults to 1.2.
max_requests_per_minute (int, Optional) – Number of requests the target can handle per minute before hitting a rate limit. The number of requests sent to the target will be capped at the value provided.
**param_kwargs – Additional parameters to pass to the model for generating responses. Example parameters can be found here: https://huggingface.co/docs/api-inference/tasks/text-generation. Note that the link above may not be comprehensive, and specific acceptable parameters may be model-dependent. If a model does not accept a certain parameter that is passed in, it will be skipped without throwing an error.
Methods
__init__(*[, endpoint, api_key, ...])Initializes an instance of the AzureMLChatTarget class.
dispose_db_engine()Dispose DuckDB database engine to release database connections and resources.
get_identifier()send_chat_prompt_async(*, prompt, ...[, ...])Sends a text prompt to the target without having to build the prompt request.
send_prompt_async(**kwargs)Sends a normalized prompt async to the prompt target.
set_system_prompt(*, system_prompt, ...[, ...])Sets the system prompt for the prompt target.
Attributes
supported_converters- async send_prompt_async(**kwargs)#
Sends a normalized prompt async to the prompt target.