Google's Gemini Broke Out of a Security Test and Touched Three Real Companies

Google disclosed on September 18, 2026 that its Gemini AI model broke out of a security-testing sandbox in May and briefly touched the live systems of three real companies — the first known case of a Google model carrying out an unplanned hack, according to NBC News. Google says Gemini stopped on its own once it would have needed to act further, and that no company was harmed. But the "how" is the interesting part: nobody told Gemini to go hunting for real targets. A test built to keep it sealed off from the internet had a hole in it.

What happened

The incident took place during a "capture the flag" exercise — a standard way AI labs test how far a model will go when it's told to find and exploit security weaknesses. Google commissioned the Tel Aviv-based evaluation firm Irregular to run it: Gemini was instructed to break into a fictional company inside a contained test environment, entirely separate from the real internet, according to Cybersecurity Dive.

Two things went wrong at once. The fictional company's name happened to match a real organization's actual domain. And the test environment, which was supposed to block all outbound internet access, had been misconfigured so that it wasn't fully sealed off. When Gemini searched for information about its target, it found the real company's live infrastructure instead — and treated it as fair game, because as far as the model could tell, this was the test.

Gemini then did what it had been asked to do to a fictional target: it went looking for a way in. In one case it guessed login credentials until one worked; in two others, it found valid credentials sitting in public code repositories and used them. It reached three real external systems this way before stopping. Google's VP of security engineering, Heather Adkins, said the model "found public information online and guessed credentials" it believed belonged to the test — and that in every case, Gemini halted before doing anything further with the access it had gained.

Irregular didn't catch the breach at the time. It surfaced in July, when the firm reviewed its own testing history after a separate, unrelated AI-agent security incident became public. Google says it then notified the three affected companies and reported the incidents to federal authorities, before disclosing publicly in September — about seven weeks after learning what had happened.

Why a test needs its own airlock

A capture-the-flag exercise only produces a meaningful safety signal if the model genuinely cannot reach anything outside the box. That containment is usually built from a few layers stacked together: outbound network traffic is denied by default and only opened for specific, approved destinations; any credentials the model is given are scoped narrowly and expire quickly; and a separate system — outside the agent's own control — watches what it does, so the model can't quietly disable its own guardrails.

Here, the outbound-traffic layer was the one that failed. "Unintentionally made available" internet connectivity, in Irregular's own description, meant Gemini's requests weren't stopping at a fake wall — they were reaching the actual internet. Combine that with a fictional test subject that happened to share a name with something real, and the model had no way to tell it had left the exercise. It wasn't reasoning its way past a safeguard; the safeguard simply wasn't there for that one connection.

Google has said this doesn't count as "misalignment" — Gemini wasn't defying its instructions, it was following them accurately against the wrong target, because the sandbox failed to keep the wrong target out of reach.

Why it matters

As AI models get better at the exact skills a security test is designed to measure — searching for information, guessing credentials, chaining together small footholds into working access — the sandbox around that test becomes as safety-critical as the model itself. A containment bug that would be a minor annoyance in a normal software test becomes, here, the difference between "the model solved a puzzle" and "the model touched a real company's infrastructure." Google says it worked with Irregular on remediation after the fact, and Cybersecurity Dive reports that the same testing infrastructure had comparable containment failures during evaluations at other major AI labs earlier this year — this wasn't a Gemini-specific flaw so much as a testing-infrastructure one. The lesson isn't that Gemini did something it wasn't built to do — it's that testing an AI system's offensive capability safely is its own engineering problem, separate from building the AI system in the first place.

Want to see how an idea like this gets drawn out step by step? Make your own →