The AI cybersecurity arms race is on

Businesses received a staggering amount of cyberattacks in June, according to Check Point, showing a rise of 20% over the previous 12 months. The breakout of AI agents from OpenAI in July to hack into the Hugging Face website, and subsequent similar events from Anthropic and Meta, indicate agentic-powered attacks will explode over the coming year.

Currently, malicious hackers have the advantage because publicly released frontier models from the US incorporate guardrails that can’t distinguish between malicious or defensive activities. As a consequence, these models default to a refusal to get involved. Hugging Face discovered this the hard way when they attempted to utilize a model to defend against the OpenAI intrusion. Their solution was to adapt a Chinese open weight model to analyze the 17,000 attack logs, find the vulnerability, and contain the intrusion.

With incidents like these happening more often, an arms race has begun with AI being both the problem and the solution.

Strength in numbers

While single agents generally perform more efficiently for well-defined tasks, research from Stanford University indicates swarms are more effective in messy scenarios with noisy data, which are more typical of unpredictable, intrusion attacks. The increased token usage by swarms raises costs, but increasingly efficient open weight models are rapidly lowering these barriers.

In the Hugging Face example, the agents worked together as a team leaving messages for each other on a message board they improvised. They shared newly found vulnerabilities, exchanged tools, and even developed conventions to address one another and to avoid overwriting each other’s work. While this may seem sinister, they were only following their designated purpose: to achieve a goal without regard to any collateral damage. We can expect bad actors to harness the power of agentic swarms through fine-tuning open weight models, and creating agents that progressively learn from their experiences.

Modern warfare has been transformed over the last four years, too, through the deployment of drones by Ukraine to defend against Russian attacks. Military strategies and the deployment of armament budgets around the world are shifting to focus on new technologies, and approaches and enterprises are now facing a similar challenge from the hostile use of agentic AI.

The drawbridge is down

As enterprises build out their own agentic systems to handle ecommerce, customer service, and marketing activities, this presents new attack surfaces for antagonistic efforts. April 2026 research from Trend Micro found almost 1,500 MCP servers directly exposed to the internet had no authentication or encryption, a rise of 200% from nine months earlier. This included 70 hosts offering direct SQL execution, and servers holding medical records.

The automation of business processes and the reduction of humans from decision making chains open up new vulnerabilities for agents with malicious intent. Arkose Labs’ 2026 agentic AI survey of 300 enterprise leaders found 97% expected an AI agent security incident within the next 12 months.

Social engineering

While agents have demonstrated their ability to break through security systems, they’re also capable of targeting humans to achieve their objectives. Recent research from Verizon indicates that 62% of successful breaches involve a human element, with phone-based attacks 40% more successful than email-based ones. In August, for instance, scammers using an AI-generated deep fake of Australian Prime Minister Anthony Albanese’s voice were able to scam investors out of $5.3 million.

If agents can break out of digital sandboxes, and generate convincing fake videos and audio, then they’re certainly capable of making basic phone calls. In July, during testing of frontier models, the UK AI Security Institute discovered an agent tried to insert malicious code into an open-source project. Attempting to get the code approved, the agent created fake online identities using them to persuade the project’s maintainer to sign it off. “This is the first time we’ve seen risks around autonomy and deception manifest this clearly without specific prompting in the real-world,” the Institute put in a write-up of the incident.

Fight AI with AI

So attackers currently have the upper hand in this escalating arms race. They have access to agents that can work around the clock, constantly probing, learning, and sharing their knowledge with other agents. They’ll only get better at this and learn ways to stay ahead of defensive systems. International agreements to delay or restrict the capabilities of frontier models won’t stop hostile actors motivated by money or rogue states pursuing other objectives. Developers and security vendors need access to the latest frontier models unfettered by restrictive guardrails if we’re to stand any chance of defending against the coming tsunami of attacks.

We can learn a lesson from recent history on this front. In 1992, the US restricted exported software to weak 40-bit encryption, citing security concerns going back to the cold war. While the US allowed stronger encryption internally, the result was weakened security for everyone as hostile antagonists were able to disrupt global supply chains that incorporated less secure software. Despite lifting the ban in 1999, embedded software containing 40-bit encryption continued to cause problems for many years across multiple countries, including the US.

Without rapid action, we may look back fondly to the world before July 2026 as a golden age for cybersecurity, a relative age of innocence.