Monitoring OAuth Applications with Azure Sentinel

For those of us who use Azure AD as their identity provider, you are probably struggling with OAuth app sprawl. On one hand it is great that your single sign on and identity is centralized to one place, but that means a whole lot of applications to monitor. When we first started leveraging Azure AD the concept of OAuth applications was really foreign to me, and though not a perfect comparison, I used to like to think of them as the cloud equivalent of on premise AD service accounts. You create an Azure AD App / Service Principal, then you grant it access – now that access can be pretty insignificant, or extremely privileged. Much like an on premise AD service account, it could be anything from read access to one folder on a file server to a local admin on every server (hope you have your eyes on that one!).

The deeper you are in the Microsoft ecosystem, the more apps you will have appear in your tenant. Users want the Survey Monkey app for Teams? Have an app. Users want to use Trello or Miro in Teams or SharePoint, have another app. The sheer number of applications can be overwhelming. Then on top of that you have any applications you are developing in house. The permissions these applications have can be either delegated permissions or application permissions (or a combination of both), and there is a massive difference between the two. A delegated permission grant is bound by what the user accessing the app can also access. So if you have an application called ‘My Custom Application’ and you grant delegated Mail.ReadWrite permissions to it, then ‘My Custom Application’ can only access the same mail items the user who signed in can (their own mailbox, perhaps some mailboxes they have been given specific access to). Application permission means that the app can access everything under that scope, so if you grant ‘My Custom Application’ application Mail.ReadWrite permissions then the application has read & write access to every mailbox in your tenant – big difference!

Consent phishing has become a real issue, as Microsoft has posted about. Users are getting better at knowing they shouldn’t enter their username and password into untrusted sites, but instead attackers may send an email that looks like its from Microsoft or come from internal and have the user consent to an application, which will be registered in your tenant and they can then use to access the users data, start looking around, find other contacts or info and away they go. For those that have Cloud App Security, it has some great OAuth controls here and you can also configure if and how users can add apps to Azure AD with user settings and admin consent.

Regardless of your policy on those settings, if you have the AuditLogs from Azure AD flowing to Azure Sentinel (you can add via the Data Connectors tab) then we can also check out all the activity in there. When you or one of your Azure AD admins creates an application under Azure AD -> App Registrations then three events will trigger in Sentinel. If you create one yourself, then search the AuditLogs table for your account to see the output.

AuditLogs
| where InitiatedBy has "youraccount@yourdomain.com"
| sort by TimeGenerated desc

First an application is added, then an owner (the person who created the app) is added to the app, then a service principal is created. If we look at the ‘Add application’ log under the TargetResources field we can see the name and id of the application created

This id corresponds to the object id in the Azure AD -> App Registrations portal

We also have a service principal created, and again we check the TargetResources under ‘Add service principal’ we can see the id displayed

This second id corresponds to the object id on the Azure AD -> Enterprise Applications portal

Confused about applications vs service principals vs object ids vs application ids? I think everyone has been at some point, thankfully Microsoft have detailed the relationships here for you. By default, when an application is created, it only has delegated user.read permissions. So the application can sign users in and read their profile and that’s it, now lets add some permissions to our app –

I have added delegated Sites.ReadWrite.All and Mail.ReadWrite – remember these are delegated permissions, so now the application can sign on users, see the users profile, and have read write access to any SharePoint Sites or mailboxes that the person who signed on can. Admin consent required = no means that you don’t require a global admin to consent to this permission for it to work, however each user who signs on will be presented a consent prompt. If an admin does consent then users won’t be prompted individually. Now, here is where things get a bit weird. When you add your permissions, you will see two entries in your logs

Now, we have added Sites.ReadWrite.All, so lets make sure that is what we are seeing in the logs.

Weird? No results. So what I have found is that when you first add permissions to an app, if no one has yet consented (either a user logging on and consenting for themselves, or an admin consenting for the tenant) the permissions are stored as EntitlementId’s

Old value = 1 EntitlementId, New Value = 3 EntitlementId’s, because we went from User.Read to User.Read, Sites.ReadWrite.All and Mail.ReadWrite. For delegated permissions there is no real way to map ids to names unfortunately.

Let’s now consent to these permissions as an admin and have a look what we can see. Push the big ‘Grant admin consent’ button on your permissions page. Now we query on actions done by ourselves and we see three new entries

So we have granted the delegated permissions from above, we added the app role assignment to my user account (so if you go to Azure AD -> Enterprise Apps -> Sentinel Test, you will now be assigned to the app and can sign into it) and then finally because we are an admin in this example, we also consented to the app for the tenant. If we dig down onto the add delegated permissions grant item, now we can see –

Now we’re talking, so we are stuck with EntitlementId’s just until someone consents, which is still a pain but we can work with that. Until either a user for themselves, or an admin for everyone consents, then no one has accessed the app. Now we can have a look at what delegated permissions have been added to apps in the last week using the ‘Add delegated permission grant’ operation.

AuditLogs
| where Category == "ApplicationManagement"
| where OperationName has "Add delegated permission grant"
| extend UpdatedPermissions = tostring(parse_json(tostring(parse_json(tostring(TargetResources[0].modifiedProperties))[0].newValue))) 
| extend AppId = tostring(TargetResources[1].id)
| project TimeGenerated, UpdatedPermissions, OperationName, AppId

Now we can see the list of delegated permissions recently added to our applications. Delegated permissions are the lesser of the two, but still shouldn’t be ignored. If an attacker tricks a user into adding an application with a lot of delegated permissions they can definitely start hunting around SharePoint, or email or a lot of other places to start working their way through your environment. You may be especially concerned with any permissions granted that have .All, which means not just access to one persons info, but anything that user can access. Much like a ‘Domain User’ from on premise AD can see a lot of your domain, a member of your Azure AD tenant can see a lot of info about your tenant too. We can hunt for any updated permissions that have All in them, we can also parse out the user who added the permissions. Depending on your policies on app registration this could be end users or IT admin staff.

AuditLogs
| where Category == "ApplicationManagement"
| where OperationName has "Add delegated permission grant"
| extend UpdatedPermissions = tostring(parse_json(tostring(parse_json(tostring(TargetResources[0].modifiedProperties))[0].newValue)))
| extend User = tostring(parse_json(tostring(InitiatedBy.user)).userPrincipalName)
| where UpdatedPermissions contains "All"
| project TimeGenerated, OperationName, UpdatedPermissions, User

Let’s remove all the permissions and go back to User.Read only (or just create a new app for testing and delete the old). Now this time let’s add some extremely high permissions, application Directory.ReadWrite.All – read and write everything in Azure AD, not bound by user permission, and the same for mail with Mail.ReadWrite – read and write all mailboxes in the tenant.

Now if we query our AuditLogs, we will find the same issue occurs with application permissions, until they are consented to, they only appear as EntitlementId’s.

Good news though! For application permissions you can query the MS Graph to find the information to map what you are after. You could store that data in a custom table, or a CSV to query against. If you search your tenant for the MS Graph application (00000003-0000-0000-c000-000000000000) then click on it and grab your ObjectID – yours will be different to mine.

Then query MS Graph https://graph.microsoft.com/v1.0/serviceprincipals/yourobjectidhere?$select=appRoles you will get an output like this, these ids are the same for all tenants. Now we have the ids, plus the friendly names and even a description.

I won’t rehash adding info from MS Graph to a custom table here, myself and heaps of others have covered that. In my case I have added all my id’s, names and descriptions to a custom AADPermissions_CL custom table so we can then query it. Now we can query the audit logs and join them to our custom table of permissions and id’s to enrich out alerting, if we get any hits on this query then we know application permissions have been added to an app (but not yet consented to)

let entitlement=
AuditLogs
| where OperationName has "Update application"
| extend AppName = tostring(TargetResources[0].displayName)
| extend AppID = tostring(TargetResources[0].id)
| extend User = tostring(parse_json(tostring(InitiatedBy.user)).userPrincipalName)
| extend Ids_ = tostring(parse_json(tostring(parse_json(tostring(TargetResources[0].modifiedProperties))[0].newValue))[0].RequiredAppPermissions)
| extend EntitlementId_ = extract_all(@"([w]{8}-[w]{4}-[w]{4}-[w]{4}-[w]{12})(b|/)", dynamic([1]), Ids_)
| extend EntitlementIds = translate('["]','',tostring(EntitlementId_))
| extend idsSplit =split(EntitlementIds , ",")
| mv-expand idsSplit
| extend idsSplit_s = tostring(idsSplit);
AADPermissions_CL
| join kind=inner entitlement on $left.PermissionID_g==$right.idsSplit_s
| project TimeGenerated, AppName, AppID, PermissionID_g, PermissionName_s, PermissionDescription_s, User

If you go ahead and consent to the permissions, and recheck the AuditLogs you can see we get two hits for ‘Add app role assignment to service principal’

And if we dig on down through the JSON we can see the permission added

Now can look back and see what application permissions have been added to our apps recently

AuditLogs
| where OperationName has "Add app role assignment to service principal"
| extend UpdatedPermission = tostring(parse_json(tostring(parse_json(tostring(TargetResources[0].modifiedProperties))[1].newValue)))
| extend AppName = tostring(parse_json(tostring(parse_json(tostring(TargetResources[0].modifiedProperties))[4].newValue)))
| extend User = tostring(parse_json(tostring(InitiatedBy.user)).userPrincipalName)
| extend AppId = tostring(TargetResources[1].id)
| project TimeGenerated, OperationName, UpdatedPermission, AppName, AppId, User

At this point you could query on particular permission sets, maybe looking for where UpdatedPermission has “ReadWrite” or anything with “All”. We can combine the two to see all delegated and application permissions added to an app with the following query, we join two queries based on the id of the application

let DelegatedPermission=
AuditLogs
| where OperationName has "Add delegated permission grant"
| extend AddedDelegatedPermission = tostring(parse_json(tostring(parse_json(tostring(TargetResources[0].modifiedProperties))[0].newValue)))
| extend AppId = tostring(TargetResources[1].id)
| project TimeGenerated, AddedDelegatedPermission, AppId;
AuditLogs
| where OperationName has "Add app role assignment to service principal"
| extend AddedApplicationPermission = tostring(parse_json(tostring(parse_json(tostring(TargetResources[0].modifiedProperties))[1].newValue)))
| extend AppName = tostring(parse_json(tostring(parse_json(tostring(TargetResources[0].modifiedProperties))[4].newValue)))
| extend User = tostring(parse_json(tostring(InitiatedBy.user)).userPrincipalName)
| extend AppId = tostring(TargetResources[1].id)
| join kind=inner DelegatedPermission on AppId
| project TimeGenerated, AppName, AddedApplicationPermission, AddedDelegatedPermission, AppId

Now that we know what events we are after, we can really start hunting. Looking for an app that had application permissions added and removed quickly, within 10 minutes, maybe someone trying to cover their tracks? We can find who added and removed the permission, which permissions and calculate the time between

let PermissionAddedAlert=
AuditLogs
| where OperationName has "Add app role assignment to service principal"
| extend UserWhoAdded = tostring(parse_json(tostring(InitiatedBy.user)).userPrincipalName)
| extend PermissionAdded = tostring(parse_json(tostring(parse_json(tostring(TargetResources[0].modifiedProperties))[1].newValue)))
| extend AppId = tostring(parse_json(tostring(parse_json(tostring(TargetResources[0].modifiedProperties))[5].newValue)))
| extend TimeAdded = TimeGenerated
| project UserWhoAdded, PermissionAdded, AppId, TimeAdded;
let PermissionRemovedAlert=
AuditLogs
| where OperationName has "Remove app role assignment from service principal"
| extend UserWhoRemoved = tostring(parse_json(tostring(InitiatedBy.user)).userPrincipalName)
| extend PermissionRemoved = tostring(parse_json(tostring(parse_json(tostring(TargetResources[0].modifiedProperties))[1].oldValue)))
| extend AppId = tostring(parse_json(tostring(parse_json(tostring(TargetResources[0].modifiedProperties))[5].newValue)))
| extend TimeRemoved = TimeGenerated
| project UserWhoRemoved, PermissionRemoved, AppId, TimeRemoved;
PermissionAddedAlert
| join kind=inner PermissionRemovedAlert on AppId
| where abs(datetime_diff('minute', TimeAdded, TimeRemoved)) <=10
| extend TimeDiff = TimeAdded - TimeRemoved
| project TimeAdded, UserWhoAdded, PermissionAdded, AppId, TimeRemoved, UserWhoRemoved, PermissionRemoved, TimeDiff

Wondering how those third party apps like Survey Monkey or a thousand other random Teams apps or OAuth apps work? Very similar in terms of hunting thankfully. For a multi tenant app, you won’t have an application object (under the Azure AD -> App Registrations portal) because that app will be in the developers tenant, but you will have a service principal (Azure AD -> Enterprise Applications portal). When you or a user consents to a third party application you will still get AuditLogs entries.

AuditLogs
| where OperationName contains "Consent to application"
| extend Consent = tostring(parse_json(tostring(parse_json(tostring(TargetResources[0].modifiedProperties))[4].newValue)))
| parse Consent with * "Scope:" PermissionsConsentedto ']' *
| extend AdminConsent = tostring(parse_json(tostring(parse_json(tostring(TargetResources[0].modifiedProperties))[0].newValue)))
| extend AppDisplayName = tostring(TargetResources[0].displayName)
| extend AppType = tostring(TargetResources[0].type)
| extend AppId = tostring(TargetResources[0].id)
| project TimeGenerated, AdminConsent, AppDisplayName, AppType, AppId, PermissionsConsentedto

It will show us whether it was an admin who consented, AdminConsent = True, or a user AdminConsent = false. Both our own apps and third party apps show under ‘Consent to application’ log items.

It is important to remember that Azure Sentinel is event driven, if you only recently enabled Azure AD Audit Logs sending to Sentinel then you may have a lot of applications already consented to with a heap of permissions, similar to how those on premise service accounts creep up in privilege over time. There are a lot of great tools out there that can audit your existing posture and help you clean up. Cloud App Security can visualize all your apps, the permissions, how common they are if you are licensed for it or use PowerShell/MS Graph to run a report. Then once you are at a known place, put in place good practices to reduce risk and be alerted –

Don’t let your users register applications. Azure AD -> User Settings. Users can register applications – set to No

Configure consent settings – Azure AD -> Enterprise Applications -> Consent and permissions. Either set ‘Do not allow user consent’ or ‘Allow user consent for apps from verified publishers, for selected permissions (Recommended)’. The latter will let users consent for applications that are classified as low risk, which by default are User.Read, openid, profile and offline_access. You can add/remove to this list of pre-approve permissions.

Configure admin consent and the appropriate workflow for IT staff to review requests above the approved permissions.

Configure Azure Sentinel to fire alerts for both new applications created, permissions added to them and consent granted to applications – that will cover you for internal apps and third party apps.

Practice least privilege for your applications, like you would on premise service accounts. If an internal team or vendor asked for a service account with Domain Admin access you would hopefully question them, do the same for application access; do you really need directory.readwrite.all, or do you just need to read & write particular users or groups? Want to access an O365 mailbox via OAuth, then don’t give access to all mailboxes – limit access with a scoping policy, and the same for SharePoint.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s