Microsoft Sentinel 101

Learning Microsoft Sentinel, one KQL error at a time

Using Logic Apps and Microsoft Sentinel to alert on expiring Azure AD Secrets — 1st Dec 2021

Using Logic Apps and Microsoft Sentinel to alert on expiring Azure AD Secrets

Azure AD app registrations are at the heart of the Microsoft Identity Platform, and Microsoft recommend you rotate secrets on them often. However, there is currently no native way to alert on secrets that are due to expire. An expired secret means the application will no longer authenticate, so you may have systems that fail when the secret expires. A number of people have come up with solutions, such as using Power Automate or deploying an app that you can configure notifications with. Another solution is using Logic Apps, KQL and Microsoft Sentinel to build a very low cost and light weight solution.

We will need two Logic Apps for our automation – the first will query Microsoft Graph to retrieve information (including password expiry dates) for all our applications, our Logic App will then push that data into a custom table in Sentinel. Next we will write a Kusto query to find apps with secrets due to expire shortly, then finally we will build a second Logic App to run our query on a schedule for us, and email a table with the apps that need new secrets.

Both our Logic Apps are really simple, the first looks as follows –

The first part of this app is to connect to the Microsoft Graph to retrieve a token for re-use. You can set your Logic App to run on a recurrence for whatever schedule makes sense, for my example I am running it daily, so we will get updated application data into Sentinel each day. Then we will need the ClientID, TenantID and Secret for an Azure AD App Registration with enough permission to retrieve application information using the ‘Get application‘ action. From the documentation we can see that Application.Read.All is the lowest privilege access we will need to give our app registration.

Our first 3 actions are just to retrieve those credentials from Azure Key Vault, if you don’t use Azure Key Vault you can just add them as variables (or however you handle secrets management). Then we call MS Graph to retrieve a token.

Put your TenantID in the URI and the ClientID and Secret in the body respectively. Then parse the JSON response using the following schema.

{
    "properties": {
        "access_token": {
            "type": "string"
        },
        "expires_in": {
            "type": "string"
        },
        "expires_on": {
            "type": "string"
        },
        "ext_expires_in": {
            "type": "string"
        },
        "not_before": {
            "type": "string"
        },
        "resource": {
            "type": "string"
        },
        "token_type": {
            "type": "string"
        }
    },
    "type": "object"
}

Now we have our token, we can re-use it to access the applications endpoint on Microsoft Graph to retrieve the details for all our applications. So use a HTTP action again, and this time use a GET action.

We connect to https://graph.microsoft.com/v1.0/applications?$select=displayName,appId,passwordCredentials

In this example we are just retrieving the displayname of our app, the application id and the password credentials. The applications endpoint has much more detail though if you wanted to include other data to enrich your logs.

One key note here is dealing with data paging in Microsoft Graph, MS Graph will only return a certain amount of data at a time, and also include a link to retrieve the next set of data, and so on until you have retrieved all the results – and with Azure AD apps you are almost certainly going to have too many to retrieve at once. Logic Apps can deal with this natively thankfully, on your ‘Retrieve App Details’ action, click the three dots in the top right and choose settings, then enable Pagination so that it knows to loop through until all the data is retrieved.

Then we parse the response once more, if you are just using displayname, appid and password credentials like I am, then the schema for your json is.

{
    "properties": {
        "value": {
            "items": {
                "properties": {
                    "appId": {
                        "type": "string"
                    },
                    "displayName": {
                        "type": "string"
                    },
                    "passwordCredentials": {
                        "items": {
                            "properties": {
                                "customKeyIdentifier": {},
                                "displayName": {
                                    "type": [
                                        "string",
                                        "null"
                                    ]
                                },
                                "endDateTime": {
                                    "type": [
                                        "string",
                                        "null"
                                    ]
                                },
                                "hint": {
                                    "type": [
                                        "string",
                                        "null"
                                    ]
                                },
                                "keyId": {
                                    "type": [
                                        "string",
                                        "null"
                                    ]
                                },
                                "secretText": {},
                                "startDateTime": {
                                    "type": [
                                        "string",
                                        "null"
                                    ]
                                }
                            },
                            "required": [
                                "customKeyIdentifier",
                                "displayName",
                                "endDateTime",
                                "hint",
                                "keyId",
                                "secretText",
                                "startDateTime"
                            ],
                            "type": [
                                "object",
                                "null",
                                "array"
                            ]
                        },
                        "type": "array"
                    }
                },
                "required": [
                    "displayName",
                    "appId",
                    "passwordCredentials"
                ],
                "type": "object"
            },
            "type": "array"
        }
    },
    "type": "object"
}

Then our last step of our first Logic App is to send the data using the Azure Log Analytics Data Collector to Microsoft Sentinel. So take each value from your JSON and send it to Sentinel, because you will have lots of apps, it will loop through each one. For this example the logs will be sent to the AzureADApps_CL table.

You can then query your AzureADApps_CL table like you would any data and you should see a list of your application display names, their app ids and any password credentials. If you are writing to that table for the first time, just give it 20 or so minutes to appear. So now we need some KQL to find which ones are expiring. If you followed this example along you can use the following query –

AzureADApps_CL
| where TimeGenerated > ago(7d)
| extend AppDisplayName = tostring(displayName_s)
| extend AppId = tostring(appId_g)
| summarize arg_max (TimeGenerated, *) by AppDisplayName
| extend Credentials = todynamic(passwordCredentials_s)
| project AppDisplayName, AppId, Credentials
| mv-expand Credentials
| extend x = todatetime(Credentials.endDateTime)
| project AppDisplayName, AppId, Credentials, x
| where x between (now()..ago(-30d))
| extend SecretName = tostring(Credentials.displayName)
| extend PasswordEndDate = format_datetime(x, 'dd-MM-yyyy [HH:mm:ss tt]')
| project AppDisplayName, AppId, SecretName, PasswordEndDate
| sort by AppDisplayName desc 

You should see an output if any applications that have a secret expiring in the next 30 days (if you have any).

Now we have our data, the second Logic App is simple, you just need it to run on whatever scheduled you like (say weekly), run the query for you against Sentinel (using the Azure Monitor Logs connector), build a simple HTML table and email it to whoever wants to know.

If you use the same Kusto query that I have then the schema for your Parse JSON action is.

{
    "properties": {
        "value": {
            "items": {
                "properties": {
                    "AppDisplayName": {
                        "type": "string"
                    },
                    "AppId": {
                        "type": "string"
                    },
                    "PasswordEndDate": {
                        "type": "string"
                    },
                    "SecretName": {
                        "type": [
                            "string",
                            "null"
                        ]
                    }
                },
                "required": [
                    "AppDisplayName",
                    "AppId",
                    "SecretName",
                    "PasswordEndDate"
                ],
                "type": "object"
            },
            "type": "array"
        }
    },
    "type": "object"
}

Now that you have that data in Microsoft Sentinel you could also run other queries against it, such as seeing how many apps you have created or removed each week, or if applications have expired secrets and no one has requested a new one; they may be inactive and can be deleted.

Using Logic Apps to version control your Azure AD Conditional Access Policies — 4th Oct 2021

Using Logic Apps to version control your Azure AD Conditional Access Policies

Azure AD Conditional Access is a fantastic tool, anyone using Azure AD as an identity provider is probably familiar with it, unfortunately like a lot of Azure AD there is no native version control though, so if you change or remove a policy there is no way to roll back. You can manage conditional access entirely through code but you may not be at that level of maturity and just want a backup available should policies be changed accidentally or maliciously.

In this example we will use Azure Sentinel to detect any new policies being created, policies being updated or policies being deleted and then store the changes in an Azure storage account. First you will need to either create a storage account or use an existing, and then create a container (folder) for your policies, I have just called mine ‘conditionalaccess’. Grab an access key for the account, we will need it in Logic Apps soon.

Your Logic App will need two sets of credentials, first an account (or Azure AD application) with enough access to read Conditional Access policies, and second the Azure Storage access key from above to write to your storage location.

Our first Logic App is just going to download all our policies from Microsoft Graph and put them into our storage account. Azure Sentinel will take over when it starts alerting on new policies, or changes and deletions, but we need to grab the current state. We could do this via PowerShell or a number of other ways, but instead we will just create a Logic App we run once – that way it will be a consistent format when we use Sentinel. Create a Logic App with recurrence as the trigger (set it to 3 months or something), hopefully this doesn’t take you that long! Our entire Logic App will look this this when done.

The first 6 steps of this are to connect to Microsoft Graph to retrieve a token for re-use. Check out this post for how to post to the Microsoft Graph to retrieve an access token. To give a little more detail to the last five steps, we first want to list the ids of our existing policies. We do that by performing a GET action on the conditionalAccess endpoint and selecting only the id.

Then the next few steps we just need to manipulate our data a little to get it into a format so we can iterate through an array and get the details of all our policies.

First we parse the JSON response from Microsoft Graph using the following schema.

{
    "properties": {
        "@@odata.context": {
            "type": "string"
        },
        "value": {
            "items": {
                "properties": {
                    "id": {
                        "type": "string"
                    }
                },
                "required": [
                    "id"
                ],
                "type": "object"
            },
            "type": "array"
        }
    },
    "type": "object"
}

Then we create an array from our list of ids using the createArray function

{
    "inputs": "@createArray(body('Parse_JSON')?['value'])"
}

Then just parse the array so we can loop through it using the following schema.

{
    "items": {
        "items": {
            "properties": {
                "id": {
                    "type": "string"
                }
            },
            "required": [
                "id"
            ],
            "type": "object"
        },
        "type": "array"
    },
    "type": "array"
}

Then finally, we are going to grab each id we pulled from Microsoft Graph, get the details of the policy (using our access token as authorization) and create a blob in our storage. When you first use the Azure Blob Storage connector you will just need to use the access key from earlier.

Then trigger your Logic App and in your storage account you should see a list of containers with your policy ids and then a json file in each with the current config.

Now, we have done all that hard work we can disable or even delete that Logic App, because we will build a new one that will keep our storage up to date when changes occur to conditional access. So create a new Logic App, this time though our trigger will be an Azure Sentinel alert. We will retrieve our account entities from the alert which we are going to use further on – we will use the entity mapping to get the operation name and the id of the policy that was changed so we know what action to take. The next few steps are exactly the same as before, post to Microsoft Graph to get a token for re-use.

We will just rename our variables to ConditionalAccessOperation and ConditionalAccessID to make it clearer, rather than using the account mapping names. Then just create a variable called CurrentDateTime using the utcNow() function, we will use this to timestamp our changes.

Then the final part we will use a control condition called ‘Switch’ where we complete different actions depending on the operation type, so for a new policy created we will do one action, for an update a different action, and for a delete a different action.

So for adding an new conditional access policy we will retrieve it from Microsoft Graph, then add a new container based on the id and a new json file with the initial policy.

For an update, it is much the same except we will add a file with the updated policy

And finally for a delete action we will just add a text file with the time stamp when it was deleted, we can’t query Microsoft Graph anymore because the policy is gone.

Your Logic App should look like this (linked for big)

Lastly, we just need to get Azure Sentinel to detect on any of these changes, so create a new analytics rule, with the following query.

AuditLogs
| where OperationName contains "conditional access policy"
| extend ConditionalAccessID = tostring(TargetResources[0].id)
| sort by TimeGenerated desc 
| project OperationName, ConditionalAccessID

Set it to run every 5 minutes, looking at the last 5 minutes of data and trigger an alert for each event (in case multiple policies are changed). For entity mappings, map them to account name and AAD User ID so that our Logic App flows properly when it is triggered.

Finally on the automated response tab under ‘Alert automation’ select the Logic App you just built. Now if you perform any of those actions you should see a lifecycle of a policy in its respective folder.

Just a couple of points-

  • Sentinel can only run every 5 minutes to search for alerts, even though it can trigger multiple times within that window. If a policy is changed, then changed again in 5 minutes when the Logic App runs it will just grab the latest update/current version.
  • If a policy is changed then deleted within 5 minutes then the changes before deletion won’t be available, because when the Logic App goes to retrieve the changes from Microsoft Graph the policy would have already been removed.

These are a couple of small trade offs to have a very inexpensive Logic App and a tiny amount of storage to backup your hard work though.