Meta's AI support tool became an account takeover route, not just a help desk shortcut. The lesson for every company using AI agents is simple: never give automation more authority than your controls can defend.
Meta now has a clearer answer to a question Instagram users were asking all week: how many people were actually hit when attackers learned to manipulate its AI-assisted recovery system. The answer is not a handful of famous accounts. It is at least 20,225 people, including 30 in Maine, according to a breach notice filed with Maine's attorney general and reported by This Week in Security.
That filing changes the shape of the story. Earlier reports focused on the surreal part, hackers allegedly asking Meta's own support chatbot to add a new email address to someone else's Instagram account, then using the reset flow to seize control. The latest disclosure makes it more concrete. Meta said the issue involved a vulnerability in an AI-assisted account recovery system for Instagram, exploited to perform password resets on user accounts.
This was not traditional phishing. No fake login page was needed. No victim had to click a link. Attackers reportedly used VPNs to appear closer to a target's region, opened a conversation with the Meta AI Support Assistant, and pushed the system into sending a reset path to an attacker-controlled email address. Once that happened, the account holder was no longer the person in charge.
The uncomfortable part is that the AI did not need to be brilliant to cause damage. It needed access. Meta's notice said the tool itself functioned as intended, but a separate code path failed to verify that the email address supplied during recovery matched the email already associated with the account. In practice, that is the difference between helpful automation and handing a stranger the front-door key.
High-profile accounts made the incident visible. Reports tied the campaign to takeovers involving the Obama-era White House Instagram handle, which had been inactive since 2017, and the account of U.S. Space Force Chief Master Sergeant John Bentivegna. Security researcher Jane Manchun Wong also said her account was taken over. Those names drew attention, but the real market was broader: short, rare, or desirable usernames that can carry serious resale value in private trading channels.
Meta said the campaign began around April 17 and continued into the week of the disclosure. Instagram began warning affected users and asking them to reset passwords and re-authenticate through secure channels. The company also said it disabled the AI chatbot for now, removed the code path that allowed the reset behavior, and began checking other chatbots across its platforms for similar problems.
That last step matters more than the patch itself. If one AI support system had enough permission to alter account recovery, other automated agents may sit near the same kind of sensitive workflow. Customer support is full of these moments: changing an email address, resetting credentials, unlocking an account, reversing a transaction, or verifying identity when a user is distressed. Those are exactly the jobs companies want AI to handle because they are costly and repetitive. They are also exactly the jobs attackers want to influence.
Founders should treat AI agents like privileged employees
For startups building AI support, this incident is a practical warning, not an abstract AI safety debate. An AI agent that can only answer questions is one thing. An AI agent that can call internal tools, change account data, or trigger recovery flows is closer to an employee with admin privileges. You would not give a new contractor unrestricted access to reset customer accounts without audit logs, approvals, and hard policy checks. A model should not get softer rules because it sounds helpful.
The minimum bar is straightforward. Sensitive actions need deterministic verification outside the model. Email changes should require confirmation through the existing trusted channel, not a new address offered in chat. Password resets should respect two-factor authentication and risk scoring. High-value accounts should have step-up review. Every AI-triggered action should leave an audit trail that a security team can inspect quickly.
Enterprise buyers should ask sharper questions as well. Vendors love to sell AI support as faster, cheaper and always available. That is not enough. Buyers need to know which internal tools the agent can call, what actions are blocked by policy, what requires human approval, how prompt-injection testing is done, and how abuse is detected after deployment. The risk is not only that a chatbot says something wrong. The risk is that it does something real.
Meta can absorb the reputational damage better than most companies, but trust does not scale automatically with infrastructure. Instagram users expected Meta's security systems to stop outsiders. In this case, attackers reportedly found the helpful system and made it do the work for them. The next phase of AI customer support will be judged less by how natural the bot sounds and more by whether companies can prove it knows when to stop.
Also read: SpaceX may wait years before the S&P 500 opens the door • Africa's startups are learning to fund growth closer to home • Andrew Bailey says AI may soon run into a power ceiling
