Poorly Thought-Out Decision will Reduce the Effectiveness and Usefulness of the Search-UnifiedAuditLog Cmdlet

On December 12, Microsoft published message center notification MC955752 to inform tenant administrators of a fundamental and problematic change in the way the Search-UnifiedAuditLog cmdlet behaves. In a nutshell, Microsoft proposes to make high completeness searches the only type of supported audit log search. That would be fine if these searches worked and were capable of finishing in a reasonable time. The problem is that high completeness searches are unreliable and take too long to complete.

Microsoft says that high completeness searches focus on retrieving a complete set of audit results in response to administrator queries. By contrast, they characterize the current behavior of the Search-UnifiedAuditLog cmdlet as speedy but can return “a subset of results.” My experience is that the current default tends to return too many results, which is why you should always sort the output set to exclude duplicates.

Microsoft then goes on to say that, out of the goodness of their hearts, they’re changing the way the Search-UnifiedAuditLog cmdlet works so that it will always run high completeness searches starting in late January 2025. To be fair, Microsoft notes that “search queries may take longer to finish.” I guess that they cannot get away from the fact that high completeness searches take between ten and twenty times longer to complete than the current searches.

A Complete Mess

I dislike the proposed change because I believe that making all audit log searches “highly complete” will affect scripts running in production today in ways that Microsoft has not anticipated.

As far as I can tell, the developers think about audit log searches as simple activities to recover audit log data. Those working with tenants use audit log searches for many other purposes, from forensic investigations following potential attacks to extracting audit results on an ongoing basis for ingestion into an external SIEM. Some ISVs use the cmdlet to extract audit information for their customers to use as the basis for some reporting.

Another example is where Microsoft engineering uses the Search-UnifiedAuditLog cmdlet to find evidence of the Exchange Web Services App Impersonation RBAC role in use. The script provided by Exchange engineering quickly becomes unusable when every iteration to find audit records takes ten minutes to complete.

Many scripts call the Search-UnifiedAuditLog cmdlet to find evidence of specific user activity. The searches might return just one or two records, enough to prove that someone is active, or they might deliver a more comprehensive assessment of the actions taken by a user account during a day. Waiting for ten minutes for a high completeness search to complete makes it impracticable for these scripts to use Search-UnifiedAuditLog (or the Graph API).

The Unreliability of High Completeness Searches

An even bigger problem is the unreliability of high completeness searches. I pointed this problem out when Microsoft released the feature in preview in April 2024. Things don’t seem to have improved much since. As an example, I ran the script to find SharePoint file operations to identify the last accessed time for documents, which is information that the Graph API for SharePoint cannot return. The search u looks for specific operations over a set period and is typical of the normal usage in Microsoft 365 tenants:

Write-Host ("Searching for SharePoint file operations in the {0} site between {1} and {2}" -f $TargetSite, $StartDate, $EndDate)
[array]$Records = Search-UnifiedAuditLog -StartDate $StartDate -EndDate $EndDate -RecordType "SharePointFileOperation" -ResultSize 5000 -SessionCommand ReturnLargeSet -Formatted -ObjectIds $TargetSearchSite
If (!($Records)) {
    Write-Host "No audit records found"
    Break
}
# Remove any duplicates
$Records = $Records | Sort-Object Identity -Unique | Sort-Object {$_.CreationDate -as [DateTime]} 
Write-Host ("{0} SharePoint Online file operations audit records found" -f $Records.Count)

Running against the site I use to store the source documents for articles, a normal search retrieved 3,879 records in 31.02 seconds. This time included the sort to remove duplicates. Running the same search with the high completeness flag set resulted in a series of failures. For example:

Measure-Command {[array]$Records = Search-UnifiedAuditLog -StartDate $StartDate -EndDate $EndDate -RecordType "SharePointFileOperation" -ResultSize 5000 -SessionCommand ReturnLargeSet -Formatted -ObjectIds $TargetSearchSite -HighCompleteness}

WARNING: Failed to process request via HighCompleteness flag, returning HttpRequestException. Exception: InternalServerError , Reason: Internal Server Error.

Days              : 0
Hours             : 0
Minutes           : 8
Seconds           : 11
Milliseconds      : 42
Ticks             : 4910423270
TotalDays         : 0.0056833602662037
TotalHours        : 0.136400646388889
TotalMinutes      : 8.18403878333333
TotalSeconds      : 491.042327
TotalMilliseconds : 491042.327

In fact, despite many attempts over several days, the search never completed. Failures happened after six, seven, or eight minutes, even during the weekend when the service experiences light demand. Using high completeness searches was a deeply unsatisfying experience that doesn’t cast a good light on Microsoft engineering or software testing.

The Microsoft Imperative to Constrain Resource Demand

Microsoft has history in messing around with the Search-UnifiedAuditLog cmdlet in ways that affect production scripts. In September 2023, they changed the way searches worked so that fewer results were returned by default. Microsoft didn’t make any announcement at the time and denied that any change had occurred even though the evidence was clear and obvious that the cmdlet now functioned in a dramatically different way.

It seems to me that Microsoft is a victim of the audit log’s success. When it was first introduced, the audit log ingested data from core workloads like Entra ID, Exchange Online, and SharePoint Online for a relatively small number of events. Eight years later, thousands of different events are ingested from multiple workloads, which increases the resources required to store and process audit events. Another extra demand for resources came when Microsoft had to up the retention period for Purview Audit standard customers from 90 to 180 days. Conserving resources is why the Audit search in the Purview portal (Figure 1) only performs asychronous searches and doesn’t use the Search-UnifiedAuditLog cmdlet.

Microsoft points to the AuditLog Query Graph API as a method for programmatic access to audit data. This is true, but setting up and running AuditLog Query searches takes substantially more effort than running Search-UnifiedAuditLog does, even using the Microsoft Graph PowerShell SDK cmdlets to interact with the API. It’s one thing for Microsoft to switch the Audit search solution in the Purview portal to use the Graph API; it’s quite another to ask customers to upgrade all their PowerShell scripts to replace Search-UnifiedAuditLog. Making this suggestion in the message center notification displays a callous lack of appreciation about how paying customers use Microsoft software.

A Problematic and Concerning Change

The lack of robustness in high completeness searches is nothing but problematic. If you can’t rely on searches completing, you can’t include search processing in scripts. Although running interactive synchronous searches obviously takes more resources, it’s how paying customers extract value from the audit log.

Changing audit searches to high completeness is bad. It might save Microsoft some computing resources, it will impact production processes by forcing Microsoft 365 tenants to use unreliable and fallible methods to access what is, after all, their audit data. Microsoft should leave well enough alone and fix high completeness to make it an optional, slow, but (when fixed) dependable alternative for audit log searches.

Insight like this doesn’t come easily. You’ve got to know the technology and understand how to look behind the scenes. Benefit from the knowledge and experience of the Office 365 for IT Pros team by subscribing to the best eBook covering Office 365 and the wider Microsoft 365 ecosystem.