AI agents built by Meta, OpenAI and Anthropic have broken into other companies’ systems in recent weeks. Developers call the incidents accidents. But of more than a dozen security experts contacted by EU Perspectives, most said the real danger lies elsewhere: human error, not rogue machines.
Most such breaches happened during testing, as AI agents from major developers tried to complete tasks and instead broke into other companies’ systems. Meta said one of its models hacked another firm after a testing partner’s error gave it unintended internet access. Anthropic said some of its models hacked three companies. OpenAI disclosed that one of its agents breached the start-up Hugging Face.
The real risk is not one hack. It is that AI agents can pursue goals across multiple steps, find vulnerabilities and adapt when blocked. Banks, healthcare providers, governments, law firms and defence-linked businesses are the most exposed. Conventional security assumes attackers are external and predictable. An AI agent, by contrast, can be an authorised internal user that still acts unpredictably. Deliberate attackers could also exploit at scale what AI agents stumble upon by accident, as one expert put it: “Anything an agent can do accidentally can be engineered deliberately.”
Errors, not machines going berserk
Sarved Dhar Badgayan, founder and group CEO of Implica Global Corporation, said the Meta incident was different from the other two. “The reported Meta incident appears to have involved a testing misconfiguration that unintentionally provided internet access; it was not necessarily a sophisticated model-driven escape from a properly secured sandbox,” he said.
The OpenAI and Anthropic cases, he added, show something more concerning. Capable agents can find weak credentials, exploit exposed services and keep pursuing a goal beyond the limits their operators intended.
You might be interested
Rich Pleeth, chief executive and co-founder of Finmile, an AI operating system for logistics and field operations, pointed to another factor: PR. “These incidents are definitely significant, but we have to remember there is also massive PR hype, look at the enormous coverage Anthropic’s Mythos received.
By announcing a model as so powerful that it is difficult to contain is of course worrying, but it is also highly effective marketing,” he said. He also pushed back on framing every incident as AI “escaping”. “Meta’s tester said a configuration error inadvertently gave the model internet access, that is a serious operational failure, but not necessarily a particularly sophisticated sandbox breakout,” he added.
Treat agents as untrusted insiders
Dr Joshua Estrin, fractional chief marketing officer at Alterion, an AI governance and runtime control company, put it more bluntly. “The messy truth is that this is not really a story about AI ‘going rogue.’ It is companies connecting increasingly capable agents to real tools, real systems, and real data before they have worked out how to contain them properly,” he said.
All three incidents, he added, share the same broader problem. “An agent does not need malicious intent to create a serious security issue. Give it a goal, tools, network access, and enough wiggle room to operate, and it may find paths that the human in the loop missed.”
The messy truth is that this is not really a story about AI ‘going rogue.’
— Joshua Estrin, Alterion
Patrick Gibbs, founder of Epiphany Dynamics, an AI automation agency that builds and deploys agentic AI systems for small and mid-sized businesses, said prevention matters more than the incident itself. “The fix isn’t a smarter model, it’s deciding in advance what the agent can never do, not just what it’s supposed to do, and building that as a hard constraint the agent can’t reason its way around. Most deployments I see define the happy path in detail and leave the boundary implicit, which is exactly where the failure shows up,” he said.
Who is legally responsible?
Badgayan had concrete advice for anyone holding sensitive data: treat AI agents as privileged, untrusted operators, not as ordinary software assistants. “An agent should never receive broad access merely because the employee using it has broad access. Credentials should be temporary, permissions task-specific, data segmented and every consequential action logged. Access to customer databases, source code, payment infrastructure, medical records or trade secrets should require an additional policy check and, at defined thresholds, human approval,” he said.
The EU has no single law covering AI agents. Instead, several pieces of legislation share the job: the AI Act, GDPR, NIS2, DORA and the Cyber Resilience Act. The AI Act can impose risk-management, cybersecurity, documentation and incident-reporting obligations, particularly for high-risk systems and general-purpose AI models that pose systemic risk. GDPR applies where personal data is accessed or exposed. NIS2 can bring cybersecurity governance and incident-reporting duties for organisations in critical sectors.
AI may execute a breach, but accountability will remain human. Proving contractual liability, negligence, trade-secret violations or unauthorised system access will still be difficult.
The winning legal strategy will not be arguing that ‘the AI did it.’
— Sarved Dhar Badgayan, Implica Global Corporation
Mr Badgayan put it plainly. “The practical problem will be attribution. Was the failure caused by the model developer, the agent provider, the company deploying it, the testing contractor, a vulnerable third party or all of them? The winning legal strategy will not be arguing that ‘the AI did it.’ Courts and regulators will ask which human organisation gave it access, failed to constrain it and ignored foreseeable risk,” he concluded.
