- Microsoft launches scanner to detect poisoned language models before deployment
- Backdoor LLMs can hide malicious behavior until specific trigger phrases appear
- The scanner identifies abnormal attention patterns linked to hidden backdoor triggers
Microsoft has announced the development of a new scanner designed to detect hidden backdoors in large, open-source language models used across enterprise environments.
The company says its tool aims to identify cases of model poisoning, a form of manipulation where malicious behavior is embedded directly into model weights during training.
These backdoors can remain dormant, allowing affected LLMs to behave normally until narrowly defined trigger conditions activate unintended responses.
How the scanner detects poisoned models
“As adoption grows, so must confidence in security measures: while testing for known behavior is relatively straightforward, the more critical challenge is building security against unknown or evolving tampering,” Microsoft said in a blog post.
The company’s AI Security team notes that the scanner relies on three observable signals that indicate the presence of poisoned models.
The first signal appears when a trigger phrase is included in a prompt, causing the model’s attentional mechanisms to isolate the trigger while reducing output randomness.
The second signal involves memorization behavior, where backdoor models leak elements of their own poisoning data, including trigger phrases, instead of relying on general training information.
The third signal shows that a single backdoor can often be activated by multiple fuzzy triggers that resemble, but do not exactly match, the original poisoning input.
“Our approach relies on two key findings,” Microsoft said in an accompanying research paper.
“First, sleeper agents tend to remember poisoning data, making it possible to leak backdoor examples using memory extraction techniques. Second, poisoned LLMs exhibit distinctive patterns in their output distributions and attention heads when backdoor triggers are present in the input.”
Microsoft explained that the scanner extracts stored content from a model, analyzes it to isolate suspicious substrings, and then scores those substrings using formalized loss functions associated with the three identified signals.
The method produces a ranked list of trigger candidates without requiring additional training or prior knowledge and works across common GPT models.
However, the scanner has limitations because it requires access to model files, meaning it cannot be used on proprietary systems.
It also works best on trigger-based backdoors that produce deterministic outputs. The company said the tool should not be treated as a universal solution.
“Unlike traditional systems with predictable paths, AI systems create multiple entry points for uncertain inputs,” said Yonatan Zunger, deputy director and deputy head of information security for artificial intelligence.
“These entry points can carry malicious content or trigger unexpected behavior.”
Follow TechRadar on Google News and add us as a preferred source to get our expert news, reviews and opinions in your feeds. Be sure to click the Follow button!
And of course you can too follow TechRadar on TikTok for news, reviews, video unboxings, and get regular updates from us on WhatsApp also.
