- AI tools are being custom-built for criminals, new GTIG report shows
- These tools side step AI bumper designed for safety
- ‘Just-in-time’ AI malware shows how criminals are developing their techniques
Google’s Threat Intelligence Group has identified a worrying shift in AI trends, where AI is no longer only used to make criminals more productive, but is also now specifically developed for active operations.
Its research found that large language models (LLMs) are particularly used in malware, with ‘Just-in-Time’ AI such as PROMPTFLUX – which is written in VBScript and works with Gemini’s API to request ‘specific VBScript obfuscation and evasion techniques to facilitate “just-in-time” self-modification that is likely to avoid static signatures.
This illustrates how criminals are experimenting with LLMs to develop ”dynamic obfuscation techniques” and target victims. The PROMPTFLUX samples examined by Google suggest that this code family is currently in the testing phase – so it could become even more dangerous as criminals develop it further.
Built for damage
The marketplace for legitimate AI tools is maturing, as is the criminal black market. Underground forums offer purpose-built AI tools that help lower the barrier for criminals to engage in illegal activities. This is bad news for everyone, as criminals no longer need to be particularly skilled to carry out complex cyber attacks, and they have a growing number of options.
Threat actors are using tactics reminiscent of social engineering to bypass AI security features – pretending to be ‘cyber security researchers’ to convince Gemini to give them information that might otherwise be off-limits.
But who is behind these incidents? Well, the research identifies, perhaps unsurprisingly, links to state-backed actors from Iran and China. These campaigns have a range of goals, from data exfiltration to reconnaissance – similar to previously observed influence operations by the states, also using AI tools.
Since AI tools have become popular, both criminals and security teams have used the tools to increase productivity and aid operations – and it’s not entirely clear who has taken the upper hand.
The best protection against identity theft for all budgets
