Entra ID tenants face threats from bad actors who use password spray attacks, multifactor spamming, and social phishing campaigns. Many organizations do not prioritize protecting Entra ID because they worry about affecting their end users. One straightforward way to protect Entra ID is to use risk based conditional access policies that combine conditional access policies with the risk signals from Entra ID Protection. In this blog, I will discuss some of the mistakes that we see organizations make that cause delays in the deployment and leave their tenants insecure. This blog will answer questions about Entra ID tenants using third party identity providers to authenticate, reducing false positives, minimizing user impact, and migrating from the old identity protection policies. 

First let us make sure a couple of things are understood.  

Entra ID protection does have a license requirement. 
A risk based conditional access policy is a conditional access policy that leverages the user or sign-in risk condition. 
When using a block for either a user risk or sign-in risk, it may require monitoring and manual remediation to be performed by at least a security operator. 
Changing password grant control will only remediate user risk and requires the user to be registered for self-service password reset or they will be blocked. 
When using cloud authentication using Azure multifactor authentication (MFA), a sign-in risk can be remediated by a multi-factor authentication, and it requires the user to be registered for MFA or they will be blocked. 
If only the required multi-factor is used with a user risk policy the user will be bombarded with MFA prompts resulting in poor user experience. 
Combining a user risk and sign-in risk in the same policy means both must be met before the policy applies. (remember an MFA prompt does not remediate the user risk) 

The workbook being referenced in this blog is designed to help show potential impact and help troubleshoot the deployment or usage of risk based conditional access policies. It requires the Entra Id signinlogs to be sent to Azure Monitor. To learn more: Impact Analysis Risk-Based Access Policies  

Issue 1: Implemented sign-in risk policies before reducing the false positives. 

One way to lower the number of false positives is to label all the IP addresses of the organization’s network egress as trusted. For assistance, administrators can go to the Entra ID’s workbooks section and find a workbook called “Impact Analysis Risk-Based Access Policies”. Open that workbook and go to the “IP addresses not listed as trusted network” section. There you can see a list of IP addresses from the existing sign-in logs where multiple users from the organization have logged in. Use the autonomous system number (ASN) to check who owns the IP ranges, decide if they are reliable and create a named location with the mark as a trusted location option. 

How to create a named location  

From the workbook, this shows all the Ip addresses that have sign-ins by multiple users. 

Some of the IP addresses in this list may belong to third party proxy solutions. If they cannot provide a source anchor IP address, they should be defined as a trusted named location. Organizations that require MFA from untrusted IP addresses should consider a separate conditional access policy that requires MFA for that new trusted named location.  Defining trusted networks is not something you do once, changes to the networks seem to occur to organizations frequently. Make sure to regularly review the report and set up new trusted networks. 

Issue 2: Implemented user risk policies prior to remediating the last several years of low, medium, and high user risk. 

I once spoke to an organization that had over 3,000 high-risk users. Think about how bad the user experience would have been if they had to change their passwords when they applied the policy. Since the day this feature was available to the tenant, Identity protection has been marking a user it determines is risky with low, medium, or high risk level. The only way to clear the risk for a user is either an administrator manually dismissing it or the user changing or resetting their password in Entra ID. This means that when a user risk policy is turned on, many users may immediately trigger the new risk based conditional access policy and have a bad user experience. And if too many users have an unpleasant experience, it usually looks like an outage and the policy is often reversed.  

There are a few things that have been added to help clean this up: 

Starting March 31st all low user risks older than 6 months will start to age out. Plan for change – Microsoft Entra ID Identity protection: “Low” risk age out  
Organization syncing password hashes, can now leverage the new Allow on-premises password change to reset user risk feature.
And a script has been published to clean out all the old user risk. GitHub: IdentityProtectionTools   

Issue 3: Make the implementation of the policies too complex. 

When I work with organizations, we usually start out with a plan to deploy the two Microsoft recommended policies: 

Require all user sign-ins to all cloud apps with medium or high sign-in risk to require multi-factor authentication.   
Require all user sign-ins to all cloud apps with high user risk to change password.  

If organizations applied these two policies, it would lower the chances of a bad actor successfully accessing the tenant. What I see is that admins get too ambitious and end up with 10+ new risk based conditional access policy scenarios that they either neglect to implement or cannot verify the actual impact and then give up. The advantage of these two scenarios is that the “Impact Analysis Risk-Based Access Policies” workbook uses existing sign-in logs and shows the number of users who signed in successfully that would have been affected if a policy were in place: 

User risk scenarios / High risk users not prompted for password change. 
Sign-in risk & trusted network scenarios / Medium or high risk sign-ins not remediated using multifactor authentication. 

From the Workbook, this shows whether the recommended policies are in place or if possible, gaps could exist. 

The plan is to begin with the basic and essential protections. Then add the more complex and tricky situations to assess. Think about tighter block scenarios for Admin portals, members with privileged roles, security information registration and requiring a compliant device when forcing a password change. 

Issue 4: Believing their third-party (federated) identity solution is all they need to protect Entra ID. 

Some objects in Entra ID are not secured by third party identity providers. These include B2B (business-to-business) guest users, poorly managed shared mailboxes, and stolen tokens used against Entra ID. Many tenants have poor hygiene and limited monitoring that allow bad actors to use cloud accounts that authenticate directly to Entra ID. To protect Entra ID from these unknown attacks, it is better to use risk based conditional access policies in addition to what your third-party solution provides. 

When the third-party identity provider (IDP) does multi-factor, the federatedIdpMfaBehavior setting should be set so that Entra ID can send the user back for MFA and the IDP can tell Entra ID that MFA was performed. 

More information about federatedIdpMfaBehavior setting. 

#AzureAD Identity Protection adds support for federated identities! 

The “Impact Analysis Risk-Based Access Policies” workbook will show if sign-in risk is currently being remediated by multifactor authentication coming from a third-party (federated) identity provider which is a fantastic way to know the policy is working. 

From the workbook, if accounts are sent back to a federated identity provider to remediate the risk, then these will not be 0: 

Issue 5: The tenant is configured to use the legacy identity protection policies. 

A change is scheduled for July 2024 that will no longer allow the changing of legacy policies. Microsoft recommends leveraging conditional access policies when applying conditions around risk, making it easier to troubleshoot and to force a sign-in frequency. If your organization is leveraging the old Identity Protection policies, it is easy to migrate over to risk based conditional access policies.  

Refer to October 2023 announcement to get information about migrating. What’s new in Microsoft Entra   

The April 2024 announcement covers timelines. What’s new in Microsoft Entra  

From the workbook, if the legacy policies are enabled these two will not be 0. 

Deploying and maintaining Entra ID Protection is crucial for organizations to protect against threats from bad actors. By avoiding common mistakes and following the best practices, organizations can effectively secure their Entra ID tenants. It is important to regularly review and update policies to ensure the continued security of the tenant. Take the first step in securing your organization by implementing risk-based conditional access policies and following the recommendations outlined in this blog. 

Thank you. 

Chad Cox 

Additional References 

Workbook: Impact analysis of risk-based access policies 

Azure AD Mailbag: Identity protection 

​Microsoft Tech Community – Latest Blogs –Read More