- AI is not too good to generate URLs – many do not exist and some may be phishing -places
- Attackers now optimize places for LLMs rather than for Google
- Developers even use unintentional dodgy URLs
New research has revealed that AI often provides incorrect URLs that could put users at risk of attacks, including phishing trials and malware.
A report from Netcraft claims that one in three (34%) login links provided by LLMS, including GPT-4.1, is not owned by the marks they were asked about 29%pointing to unregistered, inactive or parked domains and 5%pointed to non-related, but legitimates, Brand-associated domains.
Alarming, simple requests such as’ tell me the login -Websted to [brand]’led to uncertain results, which means no contradictory input was needed.
Be careful with the links that AI generates for you
Netcraft notes that this deficiency could eventually lead to widespread phishing -risks, where users easily mislead for phishing -sites just by asking a chatbot on a legitimate question.
Attackers who were aware of the vulnerability could go ahead and record non-claimed domains suggested by AI to use them for attacks, and a case in the real world has already shown confusion of AI recommending a false Wells Fargo site.
According to the report, smaller brands are more vulnerable because they are underrepresented in LLM training data, increasing the likelihood of hallucinated URLs.
Attackers have also been observed optimizing their places for LLMs rather than traditional SEO to like Google. Estimated 17,000 GitBook phishing pages aimed at crypto tractors have already been created in this way, with attackers who mimic technical support pages, documentation and login pages.
Even more worrying is that Netcraft observed developers who used AI-generated URLs in code: “We found at least five victims copying this malicious code in their own public projects-some of them show signs of being built using AI coding tools, including marker,” the team wrote.
As such, users are encouraged to verify any AI-generated content involving URLs before clicking links. It is the same kind of advice we are given to any kind of attack, with cyber criminals who use a number of attack vectors, including false ads, to get people to click on their malicious links.
One of the most effective ways to verify the authenticity of a site is to enter the URL directly in the search box instead of relying on links that can be dangerous.
