Azure AD Conditional Access is a fantastic tool, anyone using Azure AD as an identity provider is probably familiar with it, unfortunately like a lot of Azure AD there is no native version control though, so if you change or remove a policy there is no way to roll back. You can manage conditional access entirely through code but you may not be at that level of maturity and just want a backup available should policies be changed accidentally or maliciously.
In this example we will use Azure Sentinel to detect any new policies being created, policies being updated or policies being deleted and then store the changes in an Azure storage account. First you will need to either create a storage account or use an existing, and then create a container (folder) for your policies, I have just called mine ‘conditionalaccess’. Grab an access key for the account, we will need it in Logic Apps soon.
Your Logic App will need two sets of credentials, first an account (or Azure AD application) with enough access to read Conditional Access policies, and second the Azure Storage access key from above to write to your storage location.
Our first Logic App is just going to download all our policies from Microsoft Graph and put them into our storage account. Azure Sentinel will take over when it starts alerting on new policies, or changes and deletions, but we need to grab the current state. We could do this via PowerShell or a number of other ways, but instead we will just create a Logic App we run once – that way it will be a consistent format when we use Sentinel. Create a Logic App with recurrence as the trigger (set it to 3 months or something), hopefully this doesn’t take you that long! Our entire Logic App will look this this when done.
The first 6 steps of this are to connect to Microsoft Graph to retrieve a token for re-use. Check out this post for how to post to the Microsoft Graph to retrieve an access token. To give a little more detail to the last five steps, we first want to list the ids of our existing policies. We do that by performing a GET action on the conditionalAccess endpoint and selecting only the id.
Then the next few steps we just need to manipulate our data a little to get it into a format so we can iterate through an array and get the details of all our policies.
First we parse the JSON response from Microsoft Graph using the following schema.
{
"properties": {
"@@odata.context": {
"type": "string"
},
"value": {
"items": {
"properties": {
"id": {
"type": "string"
}
},
"required": [
"id"
],
"type": "object"
},
"type": "array"
}
},
"type": "object"
}Then we create an array from our list of ids using the createArray function
{
"inputs": "@createArray(body('Parse_JSON')?['value'])"
}Then just parse the array so we can loop through it using the following schema.
{
"items": {
"items": {
"properties": {
"id": {
"type": "string"
}
},
"required": [
"id"
],
"type": "object"
},
"type": "array"
},
"type": "array"
}Then finally, we are going to grab each id we pulled from Microsoft Graph, get the details of the policy (using our access token as authorization) and create a blob in our storage. When you first use the Azure Blob Storage connector you will just need to use the access key from earlier.
Then trigger your Logic App and in your storage account you should see a list of containers with your policy ids and then a json file in each with the current config.
Now, we have done all that hard work we can disable or even delete that Logic App, because we will build a new one that will keep our storage up to date when changes occur to conditional access. So create a new Logic App, this time though our trigger will be an Azure Sentinel alert. We will retrieve our account entities from the alert which we are going to use further on – we will use the entity mapping to get the operation name and the id of the policy that was changed so we know what action to take. The next few steps are exactly the same as before, post to Microsoft Graph to get a token for re-use.
We will just rename our variables to ConditionalAccessOperation and ConditionalAccessID to make it clearer, rather than using the account mapping names. Then just create a variable called CurrentDateTime using the utcNow() function, we will use this to timestamp our changes.
Then the final part we will use a control condition called ‘Switch’ where we complete different actions depending on the operation type, so for a new policy created we will do one action, for an update a different action, and for a delete a different action.
So for adding an new conditional access policy we will retrieve it from Microsoft Graph, then add a new container based on the id and a new json file with the initial policy.
For an update, it is much the same except we will add a file with the updated policy
And finally for a delete action we will just add a text file with the time stamp when it was deleted, we can’t query Microsoft Graph anymore because the policy is gone.
Your Logic App should look like this (linked for big)
Lastly, we just need to get Azure Sentinel to detect on any of these changes, so create a new analytics rule, with the following query.
AuditLogs
| where OperationName contains "conditional access policy"
| extend ConditionalAccessID = tostring(TargetResources[0].id)
| sort by TimeGenerated desc
| project OperationName, ConditionalAccessIDSet it to run every 5 minutes, looking at the last 5 minutes of data and trigger an alert for each event (in case multiple policies are changed). For entity mappings, map them to account name and AAD User ID so that our Logic App flows properly when it is triggered.
Finally on the automated response tab under ‘Alert automation’ select the Logic App you just built. Now if you perform any of those actions you should see a lifecycle of a policy in its respective folder.
Just a couple of points-
- Sentinel can only run every 5 minutes to search for alerts, even though it can trigger multiple times within that window. If a policy is changed, then changed again in 5 minutes when the Logic App runs it will just grab the latest update/current version.
- If a policy is changed then deleted within 5 minutes then the changes before deletion won’t be available, because when the Logic App goes to retrieve the changes from Microsoft Graph the policy would have already been removed.
These are a couple of small trade offs to have a very inexpensive Logic App and a tiny amount of storage to backup your hard work though.
Microsoft Sentinel 101 
