On May 11, Google’s Threat Intelligence Group published a report worth reading in full. Their researchers say they caught what they believe is the first zero-day exploit in the wild that was built with help from an AI model. The exploit was a Python script designed to bypass two-factor authentication on a popular open-source system administration tool. Google worked with the affected vendor and helped get it patched before it could be used in what the attackers had planned as a mass exploitation campaign.
The headline framing of “AI is now writing exploits” isn’t wrong, but it skips over the part with the most operational value. The technique attackers used to get the AI to help them in the first place is what’s worth learning from.
Persona-driven jailbreaking#
Most AI models have guardrails that try to prevent them from helping with obviously malicious requests. Ask one to write a working exploit for a specific piece of software, and it’ll usually refuse.
Attackers found a workaround. They tell the AI to pretend to be someone whose job it is to find vulnerabilities. A senior security auditor. A binary security expert. A researcher doing legitimate work.
Google specifically named a threat group it tracks as UNC2814 that’s been using this approach against Google’s own Gemini model. UNC2814 used persona-driven jailbreaking by directing the model to act as a senior security auditor or C/C++ binary security expert, then pointed it at TP-Link router firmware and Odette File Transfer Protocol implementations looking for vulnerabilities.
This technique has a name. Security researchers call it persona-driven jailbreaking, and it works because of a tension in how these AI systems are designed. The same model that’s trained to refuse “help me hack this device” will often comply with “you are an expert helping a colleague review their own firmware for weaknesses.” From the model’s perspective, the second framing looks like legitimate work. From the attacker’s perspective, the output is the same either way.
What the Attackers Actually Did#
The zero-day Google caught wasn’t necessarily built with persona jailbreaking. Google said it has high confidence the actor leveraged an AI model to support the discovery and weaponization of the vulnerability, citing telltale signs in the code: a hallucinated severity score, textbook Python formatting, detailed help menus, and educational docstrings characteristic of training data. The AI left fingerprints.
The exploit itself was designed for 2FA bypass on a system administration tool used widely enough that the attackers expected a mass exploitation event to be worth their time. Google declined to name the tool, which is a normal disclosure courtesy while patches roll out.
The chain of events is the part to understand. Attackers used AI to help them find a vulnerability nobody else knew about. They used AI to help them write the code that exploits it. They were planning to use it at scale. The only reason it didn’t work was that Google’s threat researchers spotted the activity early and worked with the vendor to close the hole.
For anyone newer to cybersecurity, this case illustrates how the lifecycle of an attack actually works. Vulnerability discovery, exploit development, planned deployment, and then the defender side: detection, disclosure, and remediation. The whole cycle ran in this case, just faster than usual because the attackers had AI helping them.
How Different Groups Are Using AI#
Google’s report covered more than just the one zero-day. It documents how different threat actors are integrating AI into their workflows, and the variety across groups is notable.
A threat group tracked as APT45, also known as Andariel and Onyx Sleet, sent thousands of repetitive prompts to recursively analyze CVEs and validate proof-of-concept exploits. The operational pattern is attackers using AI as a research assistant: feed it known vulnerabilities, ask it to figure out which ones can actually be exploited and how. The result is a more robust arsenal of exploit capabilities that would be impractical to manage without AI assistance, according to Google.
A threat group known as APT27 used Gemini to speed up development of a fleet management application for what’s called an operational relay box (ORB) network, which is essentially infrastructure that hides where an attack is coming from.
Threat actors targeting Ukrainian organizations have deployed AI-enabled malware called CANFAIL and LONGSTREAM, both of which use LLM-generated decoy code to conceal their malicious functionality. The malware is wrapped in fake-looking legitimate code that an AI generated to make automated security scanners miss what it’s actually doing.
Google also documented PROMPTSPY, which they describe as a shift toward genuinely autonomous attacks. PROMPTSPY is an Android backdoor that integrates Google’s Gemini API directly into its execution flow, allowing it to interpret system states and dynamically generate commands rather than relying on pre-written instructions.
PROMPTSPY is the most novel category in the report. Malware that reasons while it runs. The attacker writes the goals, and the AI figures out the steps as it goes.
The defender side#
The same report covers what Google is doing on the defensive side, which is worth flagging because the picture isn’t all bad. Google has a project called Big Sleep, an AI agent developed by Google DeepMind and Google Project Zero that actively searches and finds unknown security vulnerabilities. Big Sleep has found its first real-world security vulnerability and assisted in finding one that was imminently going to be used by threat actors, which GTIG was able to cut off beforehand. They’re also working on something called CodeMender that uses AI to automatically patch vulnerabilities.
The arms race is on. AI helps attackers find and exploit faster, AI helps defenders find and patch faster. The question for anyone getting into this field is which side moves faster, and where you want to be standing when the dust settles.
Takeaways for someone new to the field#
A few things from this story that are worth carrying forward for anyone studying for a certification, looking at cybersecurity as a career, or trying to understand what practitioners actually deal with.
First, attacks don’t happen the way movies show them. They’re not someone typing fast in a dark room. They’re patient, methodical, and increasingly assisted by tools that used to require teams of people. The work of attacking and the work of defending are both real jobs that real people do every day.
Second, social engineering isn’t going away just because AI exists. It’s expanding into new territory. The persona-driven jailbreak is social engineering against an AI model. The next attack you read about might be social engineering against an AI agent that’s been given access to a company’s data. The fundamentals around manipulation, trust, and authority apply to systems we haven’t seen yet.
Third, the defender side of this field is doing interesting work. Google’s threat researchers caught something genuinely new and helped prevent a mass exploitation event. That’s not a small thing. Threat intelligence and incident response are categories where the work is visibly meaningful for anyone trying to figure out what part of cybersecurity to get into.
Fourth, foundational concepts still matter. Reading this story requires understanding what a zero-day is, what 2FA does, what an exploit is, what a vulnerability looks like in code. The certifications that teach those fundamentals haven’t been made obsolete by AI. They’re more important than ever, because that vocabulary is what makes reports like Google’s readable in the first place.
A quote worth holding onto#
John Hultquist, chief analyst at Google Threat Intelligence Group, summed up the state of play in the report’s accompanying coverage:
“For every zero-day we can trace back to AI, there are probably many more out there. Threat actors are using AI to boost the speed, scale and sophistication of their attacks.”
That’s the honest version of where we are. Google caught one. There are others nobody has caught yet. The field is still working out the implications alongside everyone else, which is a useful environment for anyone trying to learn it.
For anyone newer to cybersecurity wondering whether it’s too late to get in, this story argues otherwise. The industry is being reshaped in real time. The people who do well in it tend to be the ones who can read a report like Google’s, understand what it actually means, and act on it.
That ability is something worth building.
