- AI hallucination could become a weapon, new report warns
- HalluSquatting is short for “adversarial hallucination squatting”
- GitHub Copilot, Gemini CLI and OpenClaw are all affected
Your favorite AI service could be subverted to deploy code that turns your phone or PC into a botnet, according to researchers at Intuit, the Technion and Tel Aviv University.
Dubbed HalluSquatting, a portmanteau of adversarial hallucinationsquatting, the technique is similar to typosquatting in that it relies on a mistake to distribute malicious code. While typosquatting can occur with the wrong entry of a website URL, HalluSquatting revolves around an LLM that is unable to identify a resource or repository with 100% accuracy.
By relying on an LLM’s tendency to hallucinate repository resource identifiers, this weakness can be scaled up to execute massive ransomware campaigns, botnets and more.
Push-me-pull-you
Previous LLM-based malware operations have relied on pull-based attacks. In this scenario, a prompt designed to jailbreak or otherwise subvert the AI is (for example) placed on a website and LLM is prompted to collect the information, thereby reducing its internal security.
What the researchers shared in their paper is that pull techniques are being combined with push attacks, which are traditionally performed as code injection.
The paper’s introductory summary states: “By preemptively detecting hallucinated resources—a technique we call adversarial hallucination squatting (HalluSquatting)—we demonstrate remote tool execution and remote code execution at scale across a variety of popular agentic LLM applications that could be leveraged to establish a botnet.”
Once an attacker has identified the resource likely to be misnamed by an LLM and squatted on it (to embed conflicting prompts), the job is done. All that is left is for a user to trigger the resource, AI chatbot or agent to initiate the response and the squatted resource will be accessed.
Promptware attack
The conflicting content stored in the squatted resource is then activated, triggering the tool invocation stage. This is the promptware attack, where attacker-controlled instructions are executed, with results potentially including turning the device you’re using into a botnet zombie.
LLMs such as Cursor, Cursor CLI, Windsurf, GitHub Copilot, Cline coding assistants have been used in the testing of this attack path along with the Gemini CLI and the OpenClaw, ZeroClaw and NanoClaw AI assistants. The researchers successfully achieved remote tool execution (essentially remote access and control of the LLMs) and remote code execution (RCE, where malicious code is executed remotely).
Some mitigations are available, including LLM developers blocking fetch operations in favor of a search tool, and resource owners enforcing strict naming, perhaps in favor of globally unique resource names. However, these will require cooperation between different parties and may take some time to implement.
The risk of LLM-based malware is increasing and some have already been seen in the wild. Of these, the JADEPUFFER attack is perhaps the most notable, as it’s not just AI-based malware – it’s a full-fledged ransomware attack powered entirely by an LLM.
Follow TechRadar on Google News and add us as a preferred source to get our expert news, reviews and opinions in your feeds.
