Reporting the Creation of SharePoint Agents

Use Audit Records to Find Who Creates SharePoint Agents

Another day, another event on the TEC 2025 European Roadshow. This time we were in Paris and just like in London’s discussion about how to protect old confidential files from Copilot access, attendees posed many good questions. Following the discussion about how to manage agents, I was asked how an organization could discover if SharePoint agents are in use. It’s a good question that isn’t answered in Microsoft’s documentation about how to manage agents in SharePoint. Microsoft gives some details about planned administrative features for agents are in an April 8, 2025 blog post, but there’s nothing available today.

I hadn’t thought about the problem up to now. SharePoint agents are limited in scope and don’t seem to pose too many administrative challenges. Each agent exists as a file in a document library with an .agent extension (originally, agents had a .copilot extension). Except for sites marked for Restricted Content Discovery, SharePoint Online sites have a default agent that reasons over the entire site. Site members can create other agents that focus on specific parts of the site. Agents created by site members are available to all site members and can be amended by them.

Approved Agents

To mark agents as being particularly useful, site owners can approve agents to highlight the agents in the agent picker. The files for approved agents are moved to the Site Assets library where they’re stored in the Approved sub-folder of the Copilots folder. Only site owners can edit approved agents. Figure 1 shows the details of an approved agent.

Details of an approved SharePoint agent. — Figure 1: Details of an approved SharePoint agent

Auditing Agent Creation

The creation and updating of custom SharePoint agents is evidence that people are using agents. Because the agent files are treated like other SharePoint files, audit records are captured in the Microsoft 365 audit log when these actions occur. By interrogating the audit log, we can discover who is creating agents and the sites where agents are used.

Here’s some PowerShell to use the Search-UnifiedAuditLog cmdlet to find SharePoint FileUploaded audit records for files with an .agent extension:

[array]$Records = Search-UnifiedAuditLog -StartDate (Get-Date).AddDays(-30) -EndDate (Get-Date) -Formatted -ObjectIds "*.agent" -Operations FileUploaded -ResultSize 5000 -SessionCommand ReturnLargeset
If ($Records) {
    $Records = $records | Sort-Object Identity -Unique
    Write-Host ("{0} audit records found" -f $Records.Count)
} Else {
    Write-Host "No audit records found"
    Break
}

$AgentReport = [System.Collections.Generic.List[Object]]::new()
ForEach ($Rec in $Records) {
    $AuditData = $Rec.AuditData | ConvertFrom-Json
 
    $ReportLine = [PSCustomObject][Ordered]@{
        TimeStamp       = Get-Date ($AuditData.CreationTime) -format 'dd-MMM-yyyy HH:mm'
        User            = $AuditData.UserId
        Action          = $AuditData.Operation
        SiteURL         = $AuditData.SiteURL
        Agent           = $AuditData.SourceFileName

    }
    $AgentReport.Add($ReportLine)
}
$AgentReport = $AgentReport | Sort-Object {$_.TimeStamp -as [datetime]} -Descending
$AgentReport | Out-GridView -Title "Custom SharePoint Agent Creation"

Write-Host ""
Write-Host "Custom agents created in these SharePoint Online sites"
$AgentReport | Group-Object SiteURL -NoElement | Sort-Object Count -Descending | Format-Table Name, Count
Write-Host ""
Write-Host "Custom agents created by these users"
$AgentReport | Group-Object User -NoElement | Sort-Object Count -Descending | Format-Table Name, Count

Figure 2 shows some sample output seen through the Out-GridView cmdlet.

Reporting audit events captured when users create SharePoint agents. — Figure 2: Reporting audit events captured when users create SharePoint agents

Some basic statistics are also produced about the sites where custom agents were created and the user accounts which create agents. To track agent usage, you can use the same technique to fetch and analyze FileAccessed audit events.

Microsoft Reports to Come?

Once again, the Microsoft 365 audit log is the source to answer questions. I’m sure that Microsoft will eventually get around to generating better-looking reports about agent creation and activity in the future. The usual course of events is that these kinds of gaps are filled sometime after functionality becomes available. Given that SharePoint agents reached general availability in November 2024, we’re still only finding out what reporting is needed for operational purposes, so it might take a while yet before we see any Microsoft reports.

Support the work of the Office 365 for IT Pros team by subscribing to the Office 365 for IT Pros eBook. Your support pays for the time we need to track, analyze, and document the changing world of Microsoft 365 and Office 365.

Cart

Cart