pyrit.prompt_target.AzureMLChatTarget

class AzureMLChatTarget(*, endpoint: str | None = None, api_key: str | None = None, chat_message_normalizer: ~pyrit.chat_message_normalizer.chat_message_normalizer.ChatMessageNormalizer = <pyrit.chat_message_normalizer.chat_message_nop.ChatMessageNop object>, max_new_tokens: int = 400, temperature: float = 1.0, top_p: float = 1.0, repetition_penalty: float = 1.0, max_requests_per_minute: int | None = None, **param_kwargs)

Bases: PromptChatTarget

A prompt target for Azure Machine Learning chat endpoints.

This class works with most chat completion Instruct models deployed on Azure AI Machine Learning Studio endpoints (including but not limited to: mistralai-Mixtral-8x7B-Instruct-v01, mistralai-Mistral-7B-Instruct-v01, Phi-3.5-MoE-instruct, Phi-3-mini-4k-instruct, Llama-3.2-3B-Instruct, and Meta-Llama-3.1-8B-Instruct).

Please create or adjust environment variables (endpoint and key) as needed for the model you are using.

__init__(*, endpoint: str | None = None, api_key: str | None = None, chat_message_normalizer: ~pyrit.chat_message_normalizer.chat_message_normalizer.ChatMessageNormalizer = <pyrit.chat_message_normalizer.chat_message_nop.ChatMessageNop object>, max_new_tokens: int = 400, temperature: float = 1.0, top_p: float = 1.0, repetition_penalty: float = 1.0, max_requests_per_minute: int | None = None, **param_kwargs) → None

Initialize an instance of the AzureMLChatTarget class.

Parameters:

• endpoint (str, Optional) – The endpoint URL for the deployed Azure ML model. Defaults to the value of the AZURE_ML_MANAGED_ENDPOINT environment variable.

• api_key (str, Optional) – The API key for accessing the Azure ML endpoint. Defaults to the value of the AZURE_ML_KEY environment variable.

• chat_message_normalizer (ChatMessageNormalizer, Optional) – The chat message normalizer. For models that do not allow system prompts such as mistralai-Mixtral-8x7B-Instruct-v01, GenericSystemSquash() can be passed in. Defaults to ChatMessageNop(), which does not alter the chat messages.

• max_new_tokens (int, Optional) – The maximum number of tokens to generate in the response. Defaults to 400.

• temperature (float, Optional) – The temperature for generating diverse responses. 1.0 is most random, 0.0 is least random. Defaults to 1.0.

• top_p (float, Optional) – The top-p value for generating diverse responses. It represents the cumulative probability of the top tokens to keep. Defaults to 1.0.

• repetition_penalty (float, Optional) – The repetition penalty for generating diverse responses. 1.0 means no penalty with a greater value (up to 2.0) meaning more penalty for repeating tokens. Defaults to 1.2.

• max_requests_per_minute (int, Optional) – Number of requests the target can handle per minute before hitting a rate limit. The number of requests sent to the target will be capped at the value provided.

• **param_kwargs – Additional parameters to pass to the model for generating responses. Example parameters can be found here: https://huggingface.co/docs/api-inference/tasks/text-generation. Note that the link above may not be comprehensive, and specific acceptable parameters may be model-dependent. If a model does not accept a certain parameter that is passed in, it will be skipped without throwing an error.

Methods

__init__(*[, endpoint, api_key, ...])	Initialize an instance of the AzureMLChatTarget class.
dispose_db_engine()	Dispose database engine to release database connections and resources.
get_identifier()	Get an identifier dictionary for this prompt target.
is_json_response_supported()	Check if the target supports JSON as a response format.
is_response_format_json(message_piece)	Check if the response format is JSON and ensure the target supports it.
send_prompt_async(**kwargs)	Send a normalized prompt async to the prompt target.
set_model_name(*, model_name)	Set the model name for this target.
set_system_prompt(*, system_prompt, ...[, ...])	Set the system prompt for the prompt target.

Attributes

api_key_environment_variable
endpoint_uri_environment_variable
supported_converters	A list of PromptConverters that are supported by the prompt target.

api_key_environment_variable: str = 'AZURE_ML_KEY'

endpoint_uri_environment_variable: str = 'AZURE_ML_MANAGED_ENDPOINT'

is_json_response_supported() → bool

Check if the target supports JSON as a response format.

Returns:

True if JSON response is supported, False otherwise.

Return type:

bool

async send_prompt_async(**kwargs)

Send a normalized prompt async to the prompt target.

Returns:

A list of message responses. Most targets return a single message,

but some (like response target with tool calls) may return multiple messages.

Return type:

list[Message]