Cloud App Security? Azure AD Identity Protection? Help!

If you are an Azure AD P2 tenant, or have E5 licensing there is a chance you have had a look at these products, the way they integrate (or don’t integrate) with each other and Azure Sentinel is sometimes a little unclear and known to change. They are meant to take the noise from your data sources like Azure AD sign in logs, or Office activity logs and make some sense of it all and direct the alerts to you, which is great. However sometimes even the alerts left over can be noisy. In Cloud App Security you can definitely tune this alerts which is helpful – for instance, you can change ‘impossible travel’ alerts to only fire on successful logons, not successful and failed. but I personally like getting as much data as I can into Sentinel and work with it in there.

The downside is that sending everything to Sentinel may mean a lot of alerts, even after Cloud App Security and Identity Protection have done their thing. Depending on the size your environment, it still may be overwhelming, say in a month you get 1430 alerts (using the below test data) for various identity issues.

You could just take the stance that for any of these you just sign the person out or force a password reset, that could result in a heap of false positives and frustrating users, and not treating more serious cases with more urgency.

When you connect Azure AD Identity Protection & Cloud App Security to Azure Sentinel, the alerts will show up in the SecurityAlert table with the ProviderNames of IPC and MCAS respectively. MCAS also alerts on a lot of other things, but we will focus on identity issues for now. When we look at the description for these alerts from Identity Protection, they are all kind of the same, something similar to “This risk event type considers past sign-in properties (e.g. device, location, network) to determine sign-ins with unfamiliar properties. The system stores properties of previous locations used by a user, and considers these “familiar”. The risk event is triggered when the sign-in occurs with properties not already in the list of familiar properties. The system has an initial learning period of 30 days, during which it does not flag any new detections…”, MCAS will give you a little more info but we need to really hunt ourselves.

To help us make sense of all these alerts, I thought we could get the details (IPv4 addresses and UserPrincipalName for this example) from our SecurityAlert, then replay that data through the Azure AD SigninLogs table and see if we can find some key alerts

let IPs=
SecurityAlert
| project TimeGenerated, Status, AlertName,CompromisedEntity,ExtendedProperties, ProviderName
| where TimeGenerated > ago (1h)
| where ProviderName in ('MCAS', 'IPC')
| where AlertName in ('Impossible travel activity','Multiple failed login attempts','Unfamiliar sign-in properties','Anonymous IP address','Atypical travel')
| where Status contains "New"
| extend Properties = tostring(parse_json(ExtendedProperties))
| extend UserPrincipalName = CompromisedEntity
| extend ipv4Addresses = extract_all(@"(([\d]{1,3}\.){3}[\d]{1,3})", dynamic([1]), Properties)
| extend ipv4Add = translate('["]','',tostring(ipv4Addresses))
| extend ipv4Split =split(ipv4Add , ",")
| mv-expand ipv4Split
| extend ipv4Split_s = tostring(ipv4Split);
SigninLogs
| project TimeGenerated, UserPrincipalName, IPAddress, AppDisplayName, ResultType, UserAgent, Location
| where TimeGenerated > ago(3d)
| where IPAddress !startswith "1.1.1."
| where ResultType == 0 or ResultType == 50158
| join kind=inner IPs on UserPrincipalName ,$left.IPAddress==$right.ipv4Split_s
| summarize AgentCount = count()by UserPrincipalName, UserAgent
| where AgentCount == 1

We get our SecurityAlerts over whatever period you want to look through, parse the IPs and UserPrincipalName data out, then we use the mv-expand operator to make a new row for each IP/UPN combination then look up that data to our SigninLogs table. Then to add some more intelligence, we exclude known trusted IP addresses (1.1.1.0/24 in the above example, you can whitelist these in MCAS too of course) and also only filter on successful (ResultType == 0) or successful and then sent to a third party security challenge, such as third party MFA (ResultType == 50158) events. We join on UserPrincipalName where we have a match on one of the IPs taken from the SecurityAlert event. Lastly we count the UserAgents used by each user and tell us when it is new, count == 1.

So get the alerts, grab the IP addresses and user, use that data to look for successful sign ins from non trusted networks on a user agent that is new to that user over the last 3 days. In my test environment full of fake data we go from 1430 alerts, to 11

I am not suggesting you just ignore the other 1119 alerts of course, but maybe these ones you prioritize higher or have a different response to.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s