Skip to content

AOAI Protocol Alignment Mode API Reference

AOAI Protocol Alignment Mode supports only gRPC protocol. This document describes the gRPC protocol for integrating with the RAI platform, detailing the events sent to the service and the events returned by the service. Each event and field is explained in detail.

Events Sent to the Service

1. Request Event

This event sends a payload to the RAI service for analysis.

Example: 

{
  "sourceType": "PROMPT",
  "apiName": "Chatcompletion",
  "payload": "{\"model\":\"gpt-4o\",\"messages\":[{\"role\":\"developer\",\"content\":\"You are a helpful assistant.\"},{\"role\":\"user\",\"content\":\"Hello!\"}]}"
}

Fields:

  • sourceType (string): The origin of the message. Possible values:

    • PROMPT
    • COMPLETION

  • apiName (string): Name of the API being invoked. Example values:

    • Chatcompletion
    • Realtime
    • Completion
    • Imagegeneration

  • payload (string): The serialized payload of the content, including model and messages.

Events Returned by the Service

messageId and contentIndex in returned events

In this mode, customer didn't specify the messageId and contentIndex when forward the request payload, we will follow below rules to return the messageId and contentIndex in the returned events.

  1. Completion API and Chat API:

  2. The prompt is collected as a whole payload, with messageId and contentIndex scoped by sourceType=Prompt.

  3. For Completion API:
    • All prompt text is concatenated into a single string.
    • Default: messageId = "0", contentIndex = 0.
  4. For Chat API:
    • messageId is the string index in the messages array.
    • contentIndex is the index of content within the message.

  5. For LLM completion, messageId is the choice index, and contentIndex is the array index of content in the choice.

  6. Real-Time API Multi-Turn Conversations:

  7. For user inputs: item_id as messageId.

  8. For model responses: len(response_id) + response_id + len(item_id) + item_id defines messageId. For example, res_01 and item_02 is 6res_017item_02.
  9. Content uses its natural indices as contentIndex.

1. Analysis Result Event

This event represents a partial annotation result for a segment of content.

Example: 

{
  "analysisResult": {
    "offset": {
      "sourceType": "PROMPT",
      "messageId": "0",
      "contentIndex": 0,
      "startOffset": 0,
      "endOffset": 50
    },
    "harmCategoryTaskResults": [
      {
        "result": "OK",
        "isBlocking": true,
        "kind": "HARM_CATEGORY",
        "harmCategoryTaskResult": {
          "harmCategory": "HATE",
          "isDetected": true,
          "severity": 3,
          "riskLevel": "HIGH"
        }
      }
    ]
  }
}

Fields:

  • offset (object): Specifies the range of content analyzed.

    • sourceType (string): Same as the input sourceType.
    • messageId (string): Same as the input messageId.
    • contentIndex (integer): Same as the input contentIndex.
    • startOffset (integer): UTF-8 start offset for the analyzed content.
    • endOffset (integer): UTF-8 end offset for the analyzed content.

  • harmCategoryTaskResults (array): Results for harm category detection tasks.

    • result (string): Indicates the outcome. Possible values:

      • OK
      • NoModel

    • isBlocking (boolean): Whether the content is blocked due to harm detection.

    • kind (string): Type of task. Example: HARM_CATEGORY.

    • harmCategoryTaskResult (object): Details of the harm detection.

      • harmCategory (string): Type of harm detected (e.g., HATE).
      • isDetected (boolean): Whether harm was detected.
      • severity (integer): Severity level (1-5).
      • riskLevel (string): Risk level (e.g., HIGH, LOW).

2. Watermark Event

This event indicates that content up to a certain offset has been analyzed and found safe.

Example: 

{
  "watermark": {
    "sourceType": "COMPLETION",
    "messageId": "0",
    "contentIndex": 0,
    "offset": 50
  }
}

Fields:

  • watermark (object):

    • sourceType (string): Same as input sourceType.
    • messageId (string): Same as input messageId.
    • contentIndex (integer): Same as input contentIndex.
    • offset (integer): Maximum offset analyzed and verified as safe.

3. Completion Event

This event indicates that the analysis for the input stream has been completed. Example: 

{
  "completion": {
    "end_reason": "END_REASON_END_OF_STREAM",
    "error_description": ""
  }
}

Fields:

  • completion (object):

    • end_reason (string): Reason for stream completion. Example: END_REASON_END_OF_STREAM.
    • error_description (string): Description of any errors encountered (empty if no errors).

Summary

The gRPC protocol facilitates seamless interaction with the RAI platform, providing robust mechanisms for real-time analysis and feedback. The events and fields detailed above serve as a comprehensive guide to understanding and implementing this integration.