Copilot Studio Agent Sends Salesforce Customer Data to Attacker

The July 7 report (“A Copilot Studio Story 2: When AIjacking Leads to Full Data Exfiltration“) from Zenity Labs is sobering reading for anyone considering how to introduce Copilot agents within a Microsoft 365 tenant. In a nutshell, Zenity created a replica of a “flagship example” of an agent created using Copilot Studio built by McKinsey & Co and proved that a an email containing a prompt injection sent to the agent could result in the generation of an emailed response containing customer data sent back to an attacker.

I have an instinctive suspicious of reports issued by security researchers because there are too many examples of overhyped text designed purely to enhance the credentials of the company. In this instance, the Microsoft Security Response Center took the issue seriously, so we should too.

The Problem is Still There

We’ve been down this path before with Copilot because researchers reported how they had compromised BizChat at sessions at the Black Hat USA conference in 2024. Copilot agents didn’t exist at the time, but the same method of sending a message for Copilot to process that convinced Copilot to do bad things was used.

Zenity reported the exploit described in the article to Microsoft, who fixed the problem in late April 2025, most likely through the deployment of a “prompt shielding mechanism.” The net result is that attackers cannot use the same avenue to exfiltrate large quantities of data. However, the kicker is that the fix works for the attack as described, but as Zenity says “Unfortunately because of the natural language nature of prompt injections blocking them using classifiers or any kind of blacklisting isn’t enough.” In other words, attackers can find new ways to use natural language prompts to convince agents to do silly things.

The Stupidity of Agents

The problem is, despite all the hype around artificial intelligence, Copilot agents are essentially stupid. They cannot distinguish between good and bad users, nor can they decide that an action demanded of them is wrong or inappropriate. As we head into an era where agents can talk to agents, the need for increased oversight about what agents do and how they do it is all too apparent.

Managing agent objects in Entra ID is a good way to incorporate agents within the infrastructure, but that doesn’t do anything to reveal what agents do in response to different user prompts, including prompts deliberately intended to do harm. You could pour over the details of Copilot interactions captured by the aiInteractionHistory API or the compliance records captured in user mailboxes by the Microsoft 365 substrate searching for evidence of attacker intervention, but what would you look for? Searching for the one API record where 500 Salesforce customer records are sent to an address in Russia might be the equivalent of seeking the proverbial needle in a haystack.

Although ISVs can work on the problem of agent governance, it’s obvious that ISVs can only work with agents using the APIs and data made available by Microsoft. Dealing with prompt injections is something that will remain a Microsoft competence.

As AI tools become more embedded into our work, the more attackers will be interested in seeking gaps. The battle between Microsoft and the bad guys to protect Copilot (apps and agents) is likely to be a ping-pong contest of exploit followed by remediation

The Goodness of Copilot Studio Agents

Don’t get me wrong. I like the ease of use that Copilot Studio brings to the agent creation process. Even an old duffer like me can create and publish an agent from Copilot Studio (Figure 1). We’re simply at the point in the evolution of AI tooling where security, governance, and management struggle to keep up with the pace of innovation and overhyped expectations.

It would be an overreaction to block users from being able to develop agents with Copilot Studio. Some controls are necessary and restricting those who develop agents to a limited group with oversight before publication seems like a reasonable step. I’m sure that more comprehensive development methodologies and structures will emerge over time and will be discussed on the web and at conferences. I’m looking forward to hearing what the experts say at the TEC event in Minneapolis at the end of September. Come along and join the debate!

So much change, all the time. It’s a challenge to stay abreast of all the updates Microsoft makes across the Microsoft 365 ecosystem. Subscribe to the Office 365 for IT Pros eBook to receive monthly insights into what happens, why it happens, and what new features and capabilities mean for your tenant.