Less than one in ten cybersecurity professionals trust AI testing tools to find vulnerabilities, and over three quarters say their AI vulnerability scanning tools missed critical bugs

Cobalt’s 2026 State of Pentesting report shows confidence in fully automated AI testing collapsed from 29% in 2025 to 9% this year
78% of respondents saw that automated tools miss critical vulnerabilities; LLM failures proved to be complex, with MTTR increasing from 19 to 36 days, and most issues remaining unresolved
Hybrid models rise to 47% adoption as experts emphasize automation must complement, not replace, elite human expertise in uncovering business logic risks

While the world praises the Mythos and the Chinese rush to create their own variant, a report that paints a completely different picture comes from Cobalt.

The cybersecurity firm just released the Cobalt State of Pentesting Report 2026, based on two comparative studies, one in 2025 and one in 2026. Polling about 450 cybersecurity professionals, Cobalt wanted to see how confident the cybersecurity community is in automated AI testing—and not so many vulnerabilities.

Last year, just under a third (29%) relied solely on AI automation for testing. This year, the figure fell to 9 per cent. Cobalt suggests that the main reason for such a steep drop in trust is the fact that 78% saw fully automated scanning tools that lack critical vulnerabilities. Another important reason is the complexity of the AI attack surface that the scanners are testing.

Context-dependent vulnerabilities

About one in three findings from an AI test are rated “high risk” – which is 2.7 times the average of conventional software, it said. At the time of analysis, less than two-fifths (38%) of the LLM vulnerabilities were also patched, while 62% remained open. The average time to resolution (MTTR) for AI/LLM security issues increased from 19 days to 36 days.

“LLM vulnerabilities are deeply context-dependent and invisible to tools that lack an architectural understanding of the application,” said Andrew Obadiaru, CISO of Cobalt. “To close the validation gap, automation should be implemented exactly where it excels, but elite human expertise remains the foundation for uncovering and mitigating the most complex business logic risks.”

It took the cybersecurity community less than a year to abandon fully automated AI testing and replace it with a hybrid model — something around 47% said they now prefer. This model is up 22% year-over-year, while the percentage of organizations using automation for low-risk environments also increased to 47%.

“While the industry is rightfully excited about the potential of Mythos-class tools, unguided algorithms are inherently prone to returning even more false positives and expensive false negatives than the automated scanners we have today,” Obadiaru continued.

Via Information security Magazine