A new experiment by cybersecurity firm Surfshark suggests that even people who consider themselves savvy online users struggle to distinguish AI bots from real people on social media.
Of the 710 participants who participated in the study conducted with graduate students from Malmö University, only 53% correctly identified more bots than the misidentified humans. This means that almost half (47%) failed the task completely.
Recent industry estimates suggest that bot-powered amplification now accounts for about 23% of political discourse on X during election seasons.
Surfshark’s own previous research found that major platforms remove more than 6.3 billion fake accounts each year, about 47 times the number of children born worldwide annually.
Even the best VPN can’t make you better at recognizing an AI-written comment, and that’s exactly the gap this experiment is trying to highlight.
The “Bot or Not” simulation puts you in the seat of a content moderator and asks a simple question: Can you really still trust your own instincts when scrolling?
Inside Surfshark’s “Bot or Not” experiment
The “Bot or Not” game is a timed, interactive simulation built by Interaction Design master’s students at Malmö University for the UNFOLD exhibition during Milan Design Week.
Players drop into a simulated social media comment section and are given 120 seconds to view 10 bot-written comments across four discussion topics.
Two of these topics were intentionally “cold,” meaning low in emotional charge: data centers and the perennial pineapple-on-pizza debate. The other two were “hot” and politically charged: immigration and women’s rights. The contrast between the four was where the most revealing data emerged.
When participants discussed data centers, they identified 71% of bots with an accuracy rate of 76%, the strongest result in the study. Pineapple on pizza was almost as good with 64% detection and 69% accuracy.
The moment the simulation moved into emotional territory, however, the performance collapsed.
At immigration, detection dropped to 54% and accuracy to 63%. On women’s rights, registration dropped to just 49%, with accuracy dropping to 61%, meaning users were both missing more bots and falsely accusing more real people of being machines.
Who struggles the most and how to take the test
The study also points to a clear “generational cliff” around the age of 40. Players up to the age of 20 were the strongest bot hunters in the dataset, finding nearly 65% of bots with more than 71% accuracy. Performance held steady through the 20s and 30s, then dropped sharply for the 41 to 50 bracket, with detection dropping to 42% and accuracy to 59%. Users over 50 did only marginally better.
According to Surfshark’s head of research Luís Costa, the takeaway is not really about literacy or media literacy in the traditional sense. The biggest blind spot the experiment revealed was emotion: When a debate gets heated, it effectively hijacks the mental “radar” people rely on to flag suspicious content.
To push back against automated deception, he argues, what users actually need is a cooler head and a better awareness of their own vulnerabilities, not sharper text analysis.
The “Bot or Not” game is now publicly available at botornot.one, and anyone can play it in their browser to see how they score against the original 710 contestants.
The broader point of the study is harder to shake off than the score of any individual playthrough. Bots are being produced by the billions, the technology that powers them is getting better at meddling, and our own emotional responses are the lever they are increasingly built to pull.
A few minutes of “Bot or Not” is a quick way to find out how often that handle is already working on you.
Follow TechRadar on Google News and add us as a preferred source to get our expert news, reviews and opinions in your feeds. Be sure to click the Follow button!
