Link Search Menu Expand Document

Event Hub Overview

Azure Event Hubs is a fully managed, real-time data ingestion service. Millions of events per second can be aggregated to build dynamic data pipelines.

We can use Event Hubs with API Management to obtain analytics of our API usage.

Create an Event Hubs namespace

An Event Hubs namespace provides a unique scoping container in which you create one or more event hubs. To create a namespace in your resource group using the Azure portal, follow these steps:

  1. In the Azure portal select Create a resource at the top left of the screen.
  2. Search for Event Hubs, then click on the resource.
    If you cannot find it there, please try the same Event Hubs term in the top search bar.

    Select Event Hubs in Portal

  3. Press Create to create the namespace, then enter the following:
    1. Select the Resource Group you created in previous labs.
    2. Enter a unique Namespace name.
    3. Select the Location you used in previous labs.
    4. Choose Basic for the Pricing Tier. To learn about differences between tiers, see Quotas and limits, Event Hubs Premium, and Event Hubs Dedicated articles.
    5. Leave the Throughput Units setting as it is. To learn more about throughput units or processing units: Event Hubs scalability.
    6. Select Review + Create at the bottom of the page, followed by Create.

      Create an Event Hub Namespace

    7. Press Go to resource.

      Event Hub Namespace Created

    8. Confirm that you see the Event Hubs Namespace page similar to the following example:

      Event Hub Namespace Home Page


Create an Event Hub

To create an event hub within the namespace, follow these steps:

  1. From the Event Hubs blade select + Event Hub>

    Add Event Hub

  2. Type a name for your event hub, then select Create.

    The partition count setting allows you to parallelize consumption across many consumers. For more information, see Partitions.

    The message retention setting specifies how long the Event Hubs service keeps data. For more information, see Event retention.

    Create Event Hub

  3. After the event hub is created, you see it in the list of event hubs.

    Event Hub Created

Create Access to the Event Hub

  1. Click on the newly-created event hub.
  2. Open the Shared access policies blade.
  3. Click on + Add.

    Event Hub Connection

  4. On the right side of your screen create a ´sendpolicy´ with just ´Send´ permissions

    Event Hub Send Policy

  5. Click on the new policy created and copy the Connection string-primary key to a notepad. Also copy the Event Hub namespace. You will use both values in the next section.

    Event Hub Connection


Create an API Management logger

Now that you have an Event Hub, the next step is to configure a Logger in your API Management service, so that it can log events to the Event Hub.

API Management loggers are configured using the API Management REST API. For this example we are going to use the “REST API Try it” Functionality to create the logger:

  1. Open the following link REST API Try It

    REST API Try It

  2. Press Sign in and use your Azure credentials that you have been using with this workshop, if prompted.
  3. Fill in the following Parameters:
    1. loggerId: eventhublogger
    2. resourceGroupName: your resource group
    3. serviceName: your Azure APIM instance
    4. subscriptionId: your Azure subscription

      RREST API Try It

  4. Replace the request Body with the following json. Make sure you replace the parameters appropriately:

     {
         "properties": {
             "loggerType": "azureEventHub",
             "description": "adding a new logger",
             "credentials": {
                 "name": "<your event hub namespace>",
                 "connectionString": "<your Connection string-primary key>"
             }
         }
     }
    
  5. Your request parameters might then look similar to this:

    Note my deviation by intential masking my SharedAccessKey. Please do not alter your key.

     {
         "properties": {
             "loggerType": "azureEventHub",
             "description": "adding a new logger",
             "credentials": {
                 "name": "eh-sjk-01062022/myeventhub",
                 "connectionString": "Endpoint=sb://eh-sjk-01062022.servicebus.windows.net/;SharedAccessKeyName=sendpolicy;SharedAccessKey=********;EntityPath=myeventhub"
             }
         }
     }
    
  6. Press Run.
  7. You should get a 201 response, confirming that the resource has been created.

    REST API Try It


Configure log-to-eventhub policies

Once your logger is configured in API Management, you can configure your log-to-eventhub policy to log the desired events. The log-to-eventhub policy can be used in either the inbound policy section or the outbound policy section.

  1. Browse to your APIM instance.
  2. Select the APIs blade.
  3. Select the API to which you want to add the policy.
    In this example, we’re adding a policy to the Echo API.
  4. Select All operations.
  5. On the top of the screen, select the Design tab.
  6. In the Inbound or Outbound processing window, enter the Code editor.
  7. Enter a new line after the <base /> tag in the inbound or outbound policy section.
  8. Select Show snippets.
  9. In the window on the right, select Advanced policies > Log to EventHub. This inserts the log-to-eventhub policy statement template.

    APIM Add Log to Event Hub

  10. Replace the policy with this snippet:

     <log-to-eventhub logger-id="<your logger id>">
         @{
             return new JObject(
                 new JProperty("EventTime", DateTime.UtcNow.ToString()),
                 new JProperty("ServiceName", context.Deployment.ServiceName),
                 new JProperty("RequestId", context.RequestId),
                 new JProperty("RequestIp", context.Request.IpAddress),
                 new JProperty("OperationName", context.Operation.Name)
             ).ToString();
         }
     </log-to-eventhub>
    
  11. Replace <your logger id> with the value you used for {loggerId} in the request URL to create the logger in the previous step (e.g. eventhublogger).

    You can use any expression that returns a string as the value for the log-to-eventhub element. In this example, a string in JSON format containing the date and time, service name, request ID, request IP address, and operation name is logged.

    APIM Add to Event Hub

  12. Click Save to save the updated policy configuration. As soon as it is saved the policy is active and events are logged to the designated Event Hub.

Verify Events are logged in Event Hub

  1. Issue a handful of test Echo API calls from within APIM (e.g. Get Retrieve Resource operation).
  2. In the Azure portal open the Event Hub you created earlier. You should see recent events. If not, give it a minute, then refresh.

    Event Hub APIM events

What to do with the data that is now in Event Hub is beyond the scope of this lab as this lab primarily focused on APIM to Event Hub integration.