Frontier AI Safety Limits Are Blocking Security Researchers Who Defend Networks

The Vetting Paradox

In June, the U.S. government placed export controls on Anthropic's Mythos and Fable models following reports that their safety mechanisms could be circumvented. The restrictions, which have since been partially lifted, highlighted a growing tension in AI deployment: the same guardrails designed to keep models out of attackers' hands are now slowing down the defenders.

Both Anthropic and OpenAI now operate invitation-only programs for cybersecurity professionals. Anthropic's Cyber Verification Program and OpenAI's Trusted Access for Cyber offer reduced restrictions to approved researchers, but only after vetting. For many in offensive security, the work of finding vulnerabilities before criminals do, this gatekeeping has become a friction point that shapes daily workflows.

Chris Anley, chief scientist at NCC Group, faces this tension directly. When his team discovers a potential software flaw, they need to verify it's exploitable before recommending a fix to clients. That verification step often requires asking an AI model to attempt an exploit. If the model refuses, citing safety policies, defenders lose a critical confirmation tool.

The problem, Anley explained, is that the same prompt can serve both attack and defense. Asking a model to "fix this code" reveals how to patch a vulnerability, but it also maps out how to exploit it. The tool cannot be cleanly divided into offensive and defensive modes. It functions more like a hammer: essential for building, but irreducibly capable of harm.

Inconsistent Enforcement

Chris Thompson, chief executive of RemoteThreat and founder of Offensive AI Con, has worked extensively inside both vetted programs. Even with approved access, he said, the guardrails behave unpredictably. A query that succeeds one day may be blocked the next, with no clear explanation. The result is that researchers spend time negotiating with the model rather than analyzing the vulnerability itself.

That inconsistency creates a practical problem: security work requires reliability. When a researcher is tracing the exploitability of a flaw, the back-and-forth with a model's refusal messages becomes a time sink that doesn't advance the analysis. Thompson noted that some researchers have begun routing around the issue entirely by switching to Chinese open-source models like GLM, which can be run locally without usage restrictions or vetting.

The irony, he argued, is that responsible researchers are being pushed away from U.S.-governed systems toward foreign-owned alternatives. If the goal of guardrails is to keep powerful tools in trusted hands, the current implementation may be achieving the opposite.

The Open-Source Escape Hatch

Paolo Stagno, chief technology officer at CrowdFense, a firm that develops and sells zero-day vulnerabilities to government agencies, described the vetted programs as treating customers like children. His team does use frontier models for reverse engineering, the process of understanding how compiled code works. But for the sensitive work of finding vulnerabilities or building exploits, they rely exclusively on open-source models run on local infrastructure.

The reason is operational security. Feeding vulnerability research into a cloud-based model risks leaking sensitive data or having it absorbed into future training runs. Open-source models, by contrast, keep the data on-premises. Stagno's approach reflects a broader pattern: when guardrails become too restrictive, researchers with the resources to do so simply move to alternatives that offer full control.

Not everyone shares that view. Giuseppe Cali, an independent security researcher who specializes in zero-days, said the guardrails haven't impeded his work because he doesn't use AI for the core steps of discovering or weaponizing bugs. He uses AI to speed up reverse engineering and to build supporting tools, tasks where refusal messages are less likely to trigger. For the actual vulnerability discovery, he prefers to keep that work manual. The game, he said, is too enjoyable to let a model play it for him.

The Vetting Bottleneck

One researcher at a smartphone-component manufacturer, who requested anonymity because he lacks authorization to speak publicly, described a different problem. His employer is not part of Anthropic's Cyber Verification Program, and as a result, the models are nearly unusable for security work. Any query that suggests security analysis triggers a refusal. The researcher said the tools are effectively locked out for his team's use case.

That anecdote points to a structural issue with the vetting model: access is contingent on approval, and approval is opaque. Mark Dowd, a veteran security researcher who has spent decades selling zero-days to Western governments, questioned whether private companies should be making those decisions at all. He noted that his work involves finding flaws and leaving them unpatched so governments can use them for intelligence operations, a practice that may bias his perspective. But the underlying question remains: who decides what qualifies as safe security research?

At DailyTechWire, we've tracked how AI companies have navigated dual-use technology dilemmas in other domains, from export controls on semiconductor tooling to restrictions on synthetic biology models. The cybersecurity vetting programs represent a similar attempt to manage risk through access control. The difference is that in cybersecurity, the line between offense and defense is not just blurry but functionally nonexistent for many tasks.

The Defender's Dilemma

Thompson framed the issue in terms of an approaching asymmetry. Attackers, he argued, will use whatever tools are available, including open-source models with no restrictions. If defenders are constrained by guardrails that attackers ignore, the gap in capability widens. He called for frontier labs to open up their programs, provide responsible access more broadly, and hold abusers accountable rather than tightening restrictions preemptively.

The alternative, he warned, is that defenders lose the AI race before it begins. The wave of attacks enabled by AI, he said, will arrive at a speed and scale unprecedented in cybersecurity history. The consulting firms and researchers working to mitigate that threat are currently being stifled by the very tools that could help them prepare.

The tension is not likely to resolve quickly. AI companies face regulatory pressure, reputational risk, and genuine uncertainty about how their models might be misused. Security researchers, meanwhile, operate in a domain where the tools of offense and defense are inseparable. The hammer problem, as Anley described it, has no clean technical solution. Every capability that helps a defender verify a flaw also helps an attacker exploit it.

What remains unclear is whether the current vetting model can adapt to that reality. If the guardrails continue to block legitimate work while failing to stop determined adversaries, the programs may end up serving neither safety nor security. The researchers we spoke with across offensive security firms, consulting giants, and independent practitioners all pointed to the same outcome: when the friction becomes too high, they route around it. The question for AI companies and policymakers is whether that routing strengthens or weakens the overall security posture of the systems everyone depends on.