- Printed words can override sensors and context in autonomous decision-making systems
- Vision language models treat public text as commands without confirming intent
- Road signs become attack vectors when AI reads language too literally
Autonomous vehicles and drones rely on vision systems that combine image recognition with language processing to interpret their surroundings, helping them read road signs, labels and markings as contextual information that supports navigation and identification.
Researchers from the University of California, Santa Cruz and Johns Hopkins set out to test whether that assumption holds when written language is deliberately manipulated.
The experiment focused on whether text visible to autonomous vehicle cameras could be misread as an instruction rather than simple environmental data, and found that large visual language models could be forced to follow commands embedded in road signs.
What the experiments revealed
In simulated driving scenarios, a self-driving car initially behaved correctly when approaching a stop sign and an active crosswalk.
When an altered sign entered the camera’s view, the same system interpreted the text as a directive and attempted to turn left despite the presence of pedestrians.
This shift occurred without any change to traffic lights, road design or human activity, indicating that written language alone influenced the decision.
This class of attack relies on indirect prompt injection, where input data is treated as a command.
The team changed words like “continue” or “turn left” using AI tools to increase the likelihood of compliance.
Language choice mattered less than expected, as commands written in English, Chinese, Spanish, and mixed language forms were all effective.
Visual presentation also played a role, with color contrast, font style and placement influencing the results.
In several cases, green backgrounds with yellow text produced consistent results across models.
The experiments compared two visual language models across driving and drone scenarios.
While many results were similar, self-driving car tests showed a large difference in success rates between models.
Drone systems proved even more predictable in their responses.
In one test, a drone correctly identified a police vehicle based on appearance alone.
Adding specific words to a generic vehicle caused the system to misidentify it as a police car belonging to a particular department, despite no physical indicators to support that claim.
All testing took place in simulated or controlled environments to avoid real-world damage.
Still, the findings raise concerns about how autonomous systems validate visual input.
Traditional security measures, such as a firewall or endpoint protection, do not address instructions embedded in physical spaces.
Malware removal is irrelevant when the attack requires only printed text, leaving the responsibility with system designers and regulators rather than end users.
Manufacturers must ensure that autonomous systems treat environmental text as contextual information rather than executable instructions.
Until these controls exist, users can protect themselves by limiting reliance on autonomous functions and maintaining manual monitoring whenever possible.
Via The register
Follow TechRadar on Google News and add us as a preferred source to get our expert news, reviews and opinions in your feeds. Be sure to click the Follow button!
And of course you can too follow TechRadar on TikTok for news, reviews, video unboxings, and get regular updates from us on WhatsApp also.
