There is a need for organizations to rethink their monitoring strategy when components are more and more in a cloud. Currently, there are multiple ways to pull security-related monitoring data from the Microsoft cloud and this field has been constantly changing.

The Azure Log Integration has been in place and it is a Windows operating system service that you can use to integrate raw logs from your Azure resources into your on-premises security information and event management (SIEM) systems. The downside is that Microsoft has been announced that The Azure Log integration feature will be deprecated by 06/01/2019. AzLog downloads will be disabled by Jun 27, 2018. One way to get security related data could be to use Microsoft Security Graph, more information from here and here.

Naturally, when Log integration is going to be deprecated there is a new solution available, stream Azure security data to Azure monitoring, then to Event Hub and from there to SIEM system.

Pipeline.PNG

Data can be sent to Event Hub from the following Azure layers (docs.microsoft.com):

  • Application Monitoring Data
    • Application performance data which include performance traces, application logs, and user telemetry
  • Guest OS monitoring data
    • Windows system events and Linux Sysvol
  • Azure resource monitoring data
    • Data about the operation of an Azure resource. For some Azure resource types, such as virtual machines, there is a guest OS and application(s) to monitor inside of that Azure service. For other Azure resources, such as Network Security Groups, the resource monitoring data is the highest tier of data available (since there is no guest OS or application running in those resources)
  • Azure subscription monitoring data
    • Data about the operation and management of an Azure subscription, as well as data about the health and operation of Azure itself. The activity log contains most subscription monitoring data, such as service health incidents and Azure Resource Manager audits. You can collect this data using a Log Profile.
  •  Azure tenant monitoring data
    • Data about the operation of tenant-level Azure services, such as Azure Active Directory. The Azure Active Directory audits and sign-ins are examples of tenant monitoring data. This data can be collected using a tenant diagnostic setting

The idea is that data is sent to Event Hub where SIEM system pulls the data into SIEM tool.

Set up Event Hub

Deploying Event Hub is something I’m not going to cover it in this blog post.  Naturally, there is some variation in configurations depending on what data you are sending to the Hub. Guidance on how to create Event Hub can be found from here: Creating Azure Event Hub.

The aim is to send Azure AD and Azure Security Center data to Event Hub, basically most security related stuff to SIEM. Because processed events produced by Azure Security Center are published to the Azure Activity log I needed to send data from categories Azure subscription and Azure tenant monitoring data.

Stream Azure Activity Log to Event Hub

Navigate to Azure Monitor – Activity Log and select export

6

Configure the following settings:

Subscription name: <name of your subscription>

Regions: Select all (Activity Log is global log so most event doesn’t have region associated to them)

Select event hub namespace: <namespace name>

Select event hub policy name: RootManaeSharedAccessKey – if using the default policy

7.PNG

As Microsoft states:

For the Azure Activity Log, Azure Monitor creates an event hub within that namespace called ‘insights-logs–operationallogs.‘ For all other log types, you can either choose an existing event hub (allowing you to reuse the same insights-logs-operationallogs event hub) or have Azure Monitor create an event hub per log category.

When integration has been successful insights-logs-operationallogs can be seen in the Event Hub service

10

 

Stream Azure tenant monitoring data to Event Hub

  • Navigate to Azure AD portal and there to Sign-in or Audit Logs section.
  • Select “Export Data Settings”

8

  • Configure diagnostics settings for Azure AD
    • Name:
    • Archive to storage account: this is optional, I’m sending data to storage account and Event Hub
    • Event Hub details: subscription, namespacename, event hub name and policy which is used to access event hub
    • Log: Select which logs you want to send and retention time for data in Event Hub

9

When configured is finished you can track Event Hub metrics from the service and verify that messages are coming through. In real production environments, I’m also configuring Event Hub to send its own monitoring data from diagnostics to a storage account and Azure Log Analytics. By doing so I can verify Event Hub functionality and metrics easily.

11.png

Summary

Sending Azure components monitoring data to Event Hub is a new way to do integration to SIEM system and definitely needs to be in place. Because Log Integration feature will be deprecated next summer I encourage you to test SIEM integration with Azure via Event Hub as soon as possible.  Hope this helps to make smooth integration in Azure side. Because I’m not a monitoring specialist I cannot provide instructions of SIEM system configurations.

Here is a starting point anyway:

IBM QRadar integration to Azure Event Hub

Splunk integration to Azure Event Hub