What happens when your employees’ AI agents start coming to work? (Spoiler: It’s bad)

fastcompany.com

Personal AI agents are having a moment. Meta recently launched a personal agent it named Muse and saw an estimated 2.8 million installs in its first 12 days , outpacing ChatGPT’s own mobile debut. Meanwhile, Instinct, a startup developing a personal agent you can text or call, just raised $1 billion at a $10 billion valuation . SpaceXAI introduced its Grok Bot agent in August, and most recently, on September 29, OpenAI released Dots , agents built to pursue goals in the background with minimal oversight. So, what are personal agents? A simple way to understand them is to think of them as employees that are always on. You give them a goal, you give them the access and tools they need to achieve that goal, and then you get out of the way and let them get to work. The key new piece of the puzzle , the thing that is driving the hype, is that these goals are real-world goals—things like shopping, making dinner reservations, replying to emails, filing insurance claims, and so on. The potential productivity gains are enormous. But there’s no such thing as a free lunch, and the upside comes with serious risks that business leaders must urgently address. This piece tells you what you need to do. The risk-reward trade-off The value of a personal agent is directly related to the access and permissions you give it: The more an agent knows about your relationships, schedule, and spending, the more useful it becomes. For example, it would be incredibly useful for frequent travelers to have their flights and hotels organized by agents, but this is only possible if the agent can check calendars, write messages, and make payments. But this same access can make agents dangerous. In February, for instance (and somewhat ironically given how hard Meta has pushed Muse), Meta’s own director of AI alignment watched an agent she had connected to her inbox start deleting it , ignoring her instruction to check with her first . The point here is not that the risk is so great that we should never use personal agents. The point is rather that the risk cannot be separated from the reward. An agent that can’t read your inbox can’t clear it, and one that can’t spend money can’t book your flight. Every permission that makes an agent more useful also widens what can go wrong. Personal agents will come to work The issue for enterprise is that the access people grant doesn’t stop at their personal lives. In a recent survey of knowledge workers, employees reported giving AI tools direct access to company email, cloud storage, and customer databases, and more than half of those using unapproved AI tools said they had shared internal messages and emails with them. And leaders may be overestimating how much of this is visible and being managed—in the same survey, 90% of executives said they were confident in their visibility into AI tools, while 52% of employees admitted to using AI tools without approval. Meanwhile, a Cloud Security Alliance survey found that 82% of organizations have AI agents running in their systems that they don’t know about, and 65% had suffered an agent-related security incident in the previous year. Jim Taylor, president of the security company RSA, describes learning about a midsize global bank that believed it had no AI agents, only to find out it was running more than 4,000. These surveys cover AI tools and agents of all kinds used at work, but they don’t isolate personal agents such as Muse and Dots; indeed, the surveys were done before these products became widely available. But the implications for personal agents and enterprise are clear and stark—people are already hooking up their personal agents to their company emails and databases. When they do, the agent’s risks become the company’s risks. 3 risks leaders need to understand Here are three of the most important risks that come with personal agents. 1. Your data leaves the building. This risk needs no attacker. Connect a personal agent to work email, and copies of company data go to a vendor your company has no contract with, sitting outside its procurement and compliance rules. Muse users, for instance, must opt out if they don’t want their interactions used to train Meta’s models, and we can be certain that many will forget to do so when using Muse in work contexts. 2. Outsiders can give it orders. Anyone who can email your employee can try to inject their agent instructions. One early Instinct tester emailed his own inbox from a brand-new Gmail account with instructions meant for the agent. After seeing how easily it could be phished , he deleted his account. Even Meta, which built layers of defenses into Muse, says “ prompt injection remains an open problem in the industry .” 3. It’s a master key. Whoever takes over an agent inherits everything it’s connected to. Last month, a researcher showed that any app or command running on a Mac could hijack Muse and the permissions its owner had granted , and steal the token that logs into the user’s Muse account. Agents can also read the inbox where security codes arrive: Instinct was observed pulling a sign-up code from a tester’s inbox to finish a restaurant booking. In the wrong hands, that ability could unlock accounts the agent was never connected to. The first risk can largely be handled through vendor choice and settings. The other two risks can’t be removed without giving up what makes agents useful—but they can be reduced. In all three cases, the key is setting clear limits on what agents can touch and do. 4 things to do now Each of the steps below targets one or more of the three key risks. None requires new software, and all can start this week. 1. Find every agent this week. Since leaders may see less than they think, start by looking. Ask IT to pull the list of third-party apps connected to company email and calendar accounts. At the same time, send staff a short, no-blame survey: “Which AI agents do you use, and what have you connected them to?” By the end of the week, you should have one initial list. 2. Publish one page of agent rules. This is where you set the limits. Sort agents into three lists: approved, approved with limits, and banned. “Approved with limits” might mean authorized for use with personal accounts only. Ban connecting any agent to finance systems, customer data, password managers, or the inbox where security codes arrive; those are the connections that turn an agent into a master key. Approve only vendors that let you delete your data and opt out of training. Name one person who owns the rules page, and update it every month. 3. Require a human “yes” before any action. Because outsiders can try to steer an agent, keep a person between it and anything consequential. Agents may read, sort, and draft. But a person must approve before an agent sends email outside the company, spends money, shares a file, or accepts terms. And where possible, give the agent its own email address and forward it only what it needs, instead of opening up your whole inbox. 4. Practice switching agents off. Data can outlive the connection, so test whether disconnecting actually works. Pick one agent. Revoke its account access, change any passwords it used, and cancel any payment card linked to it. Twenty-four hours later, check whether it is still acting or still holding data. Ask the vendor in writing to delete your information. Then add “disconnect all agents” to the checklist you use when employees leave the company. Set the terms now Personal agents will soon be as ordinary as smartphones. Indeed, the problem reminds me of the bring-your-own-device problem of the 2010s, when employees started using their personal phones for work email. But there is a critical difference here: A phone could store your company’s data—an agent can act on it. Just as people were using their own phones at work a decade ago, your people are already using these agents. Either you set the terms on which personal agents enter your business, or you let others do it for you.