Cowritten By: @suhelp and @arishojaswi 

To receive the highest quality responses from Copilot, users have the option to allow Copilot to reference web content. Querying the web improves the quality of Copilot responses by grounding them in the latest information from the web. Throughout the process the prompt and response remain within the service boundary; the queries Copilot generates have the user and tenant identifiers removed and are sent securely to the Bing search service.  

Screenshot: Web content plugin end user toggle for Microsoft 365 Copilot

As more customers experience the benefits of web grounding, we’ve heard a strong interest in being able to see the queries Copilot generates to fetch that information from the Bing search service. Today, we’re excited to announce new features that will allow both users and admins greater visibility into Copilot generated web queries. 

Bing web search query citations for users 

Web search query citations for users (generally available in October) will include the exact web search queries derived from the user’s prompt in the linked citation section of the Copilot response, helping users understand what search queries, along with the sites searched (available today), were used to enhance the response. 

This provides users with valuable feedback as to exactly how their prompts are used to generate web queries that are sent over to Bing search. This transparency helps users with the information they need to improve prompts and use Copilot more effectively. 

To learn more about data, privacy, and security for web queries in Microsoft 365 Copilot, please refer to the Microsoft Learn article Data, privacy, and security for web queries in Microsoft 365 Copilot. 

Web search queries will be displayed in the linked citation section of the Copilot response

Audit logging and eDiscovery for Bing web search queries   

Web search query logging (generally available in Q4) will enable admins to perform search, audit, and eDiscovery on the exact web search queries Copilot derived from the user’s prompt. Admins can already perform these actions for prompts and responses and will be able to use their familiar tools to extend those actions to search queries.* 

Providing transparency into web queries supports our mission of Trustworthy AI, as customers can confirm that the information searched by Copilot is relevant and appropriate. 

Admins will be able to use the audit log to find events involving web queries. Within the CopilotInteraction audit log, there is a property AISystemPlugin.Id which can be used to identify whether a specific Copilot interaction had a query sent to Bing. If it did, then AISystemPlugin.Id will contain the value ‘BingWebSearch’.  

In order to identify all Copilot interactions where a web search query was generated, admins will first filter and export all relevant CopilotInteraction audit logs. This can be done by filtering operation = CopilotInteraction. Once the search results are exported, admind can then filter by AISystemPlugin.Id offline.  

Admins will be able to use the audit log to find events involving web queries

For eDiscovery, the Bing search query is stored in the user’s Exchange mailbox as part of the Copilot response in the form of a metadata. Admins and eDiscovery Manager can search for the prompts and responses using Copilot Activity filter in eDiscovery condition builder and view the detailed conversations including the Bing search queries. Additionally, the CopilotInteraction audit record contains a messageId which points to this response message, which can be retrieved using eDiscovery. Learn more about using eDiscovery in the Microsoft Learn article Search for and delete Microsoft Copilot for Microsoft 365 data. 

Our continuing commitment to enhancing web grounding control 

As the examples above illustrate, we’re continuing to invest in giving customers increased visibility and control over web grounding for Copilot. To learn more, please see the Microsoft Learn article How Microsoft handles generated queries.  

*Note: The specific controls will vary depending on the underlying subscription plan. 

​Microsoft Tech Community – Latest Blogs –Read More