Before summer Microsoft launched new Azure AD monitoring capabilities, “Workbooks” and “Usage & Insights” which are visible at the Azure AD portal.

In this blog post, I’m going through the fundamentals of “Usage & Insights” and some scenarios where “workbooks” can be useful from a monitoring point of view (with a twist of Sentinel).

Usage and Insights

Usage and Insights blade contain detailed information from application usage and authentication methods activity from the tenant. This feature is quite new and for some reason, many people I have met don’t even know that it exists.

Application Usage

You can get a list from application usage easily out from the tenant. If you find something unusual from “view sign-in activity” you can dig to more detailed information from the application sign-ins.

I personally, use this information during Azure AD security checks to find unusual activities. For example, it’s quite easily seen if Azure AD is being hammered with false logins from here.

Depending on is the application first-party Microsoft Application or 3th party application you will get a bit different application description.

  • Microsoft App – description opens
  • 3th party App – Application blade opens

Authentication Methods Activity

No more multiple Powershell queries to get authentication methods activity, even I like them a lot:) Now you can get reports straight to Azure AD monitoring blade and without Global Admin permissions, Azure AD Security Reader is enough based on my tests.

Each individual report contains very good detailed information which earlier needed to grab out from Azure AD via PowerShell. Below is an example from MFA & SSPR registration reports (registration methods included). Quite nice:)

Workbooks

Default Workbooks

There are very useful default workbooks available already:

  • Sign-ins – all sign-ins, success, pending actions, failures
  • Legacy autH – all sign-ins, success
  • Sign-ins by Conditional Access
  • Sign-ins by grant controls – Require MFA, Require Terms of use, Require privacy statement

Custom Workbooks

Is available, of course:) So, your own imagination comes here into play. In the following example, I will create a workbook for monitor Service Principals with a twist of Azure Alerts & Sentinel based on a request I received last week.

The request was: monitor the following Azure AD Core Directory actions:

  • Service Principals creations
  • Service Principals modifications

How to create custom Workbook

From the Workbook blade, select the “Empty” workbook.

Define a query for the data you want to find from the Log Analytics workspace. In my example, I’m using one of the simplest ones I figured out.

When you have a query in place, run it and results will come visible. When you are satisfied with your results save the query to the wanted location.

After saving the Workbook, you can see your custom Workbook on the blade start page where workbooks can be easily executed.

Azure Alert

Now the queries are defined. The next step is to create Azure Alert to get information if someone creates or modifies Service Principal.

In my case, I have defined the query in the workbook and verified the results.

Run once that same query in Log Analytics. In Log Analytics, the query can be saved (which I see quite useful). From Log Analytics you can create Azure Alert from the queries you have used.

In a nutshell, create the query, verify results, create Azure Alert with needed parameters from the query, enjoy automatic monitoring 🙂

Azure Sentinel

Why Azure Sentinel is mentioned in here? Sentinel is built top of Log Analytics and if you have Azure AD connector in place the data can be seen in Sentinel also (relations to Log Analytics architecture – where data is stored).

In Sentinel, I created a use case (Analytics) with “Low” status to get status from Service Principals also to my cloud-based SIEM system.

Testing

Time to test does this work as expected. I created a new Service Principal to my directory, added Box application from Azure AD gallery. After 5min two events were found from Directory logs. Time of creation 08/19/2019 10:13AM.

5min (total 10min) more of waiting, I can see the same information in Log Analytics.

Navigated to the Azure Sentinel blade where I found two (2) recent incidents from this same topic. It’s not a duplicate, rather two separate ones because I created two different Apps & Service Principals to my tenant on the same morning.

References

Usage and Insights reports in Azure AD

How to use Azure Monitor workbooks for AAD reports

How to detect threats in Sentinel

Thin concludes this blog post. Hopefully, you will find this useful when planning Azure AD monitoring! Until next time 🙂