pyrit.prompt_target.HuggingFaceEndpointTarget

pyrit.prompt_target.HuggingFaceEndpointTarget#

class HuggingFaceEndpointTarget(*, hf_token: str, endpoint: str, model_id: str, max_tokens: int = 400, temperature: float = 1.0, top_p: float = 1.0, verbose: bool = False)[source]#

Bases: PromptTarget

The HuggingFaceEndpointTarget interacts with HuggingFace models hosted on cloud endpoints.

Inherits from PromptTarget to comply with the current design standards.

__init__(*, hf_token: str, endpoint: str, model_id: str, max_tokens: int = 400, temperature: float = 1.0, top_p: float = 1.0, verbose: bool = False) → None[source]#

Initializes the HuggingFaceEndpointTarget with API credentials and model parameters.

Parameters:

  • hf_token (str) – The Hugging Face token for authenticating with the Hugging Face endpoint.

  • endpoint (str) – The endpoint URL for the Hugging Face model.

  • model_id (str) – The model ID to be used at the endpoint.

  • max_tokens (int, Optional) – The maximum number of tokens to generate. Defaults to 400.

  • temperature (float, Optional) – The sampling temperature to use. Defaults to 1.0.

  • top_p (float, Optional) – The cumulative probability for nucleus sampling. Defaults to 1.0.

  • verbose (bool, Optional) – Flag to enable verbose logging. Defaults to False.

Methods

__init__(*, hf_token, endpoint, model_id[, ...])

Initializes the HuggingFaceEndpointTarget with API credentials and model parameters.

dispose_db_engine()

Dispose DuckDB database engine to release database connections and resources.

get_identifier()

send_prompt_async(*, prompt_request)

Sends a normalized prompt asynchronously to a cloud-based HuggingFace model endpoint.

Attributes

supported_converters

async send_prompt_async(*, prompt_request: PromptRequestResponse) → PromptRequestResponse[source]#

Sends a normalized prompt asynchronously to a cloud-based HuggingFace model endpoint.

Parameters:

  • prompt_request (PromptRequestResponse) – The prompt request containing the input data and associated details

  • role. (such as conversation ID and)

Returns:

A response object containing generated text pieces as a list of PromptRequestPiece

objects. Each PromptRequestPiece includes the generated text and relevant information such as conversation ID, role, and any additional response attributes.

Return type:

PromptRequestResponse

Raises:

  • ValueError – If the response from the Hugging Face API is not successful.

  • Exception – If an error occurs during the HTTP request to the Hugging Face endpoint.