- A new Runway study asks volunteers to identify AI videos
- Clips were correctly identified only 57.1% of the time
- The videos were generated with the new Runway Gen-4.5 model
It seems we’re through the looking glass when it comes to the veracity of AI-generated videos: In a new test conducted by AI video company Runway involving more than 1,000 participants, volunteers were only able to correctly identify AI or real videos 57.1% of the time.
Considering that guessing randomly would land you somewhere around the 50% mark, that’s a worryingly low number. Even Runaway co-founder and Chief Technology Officer Anastasis Germanidis admitted to failing “quite a bit” at the task (via The Information).
If Runway’s own team members struggle, the rest of us don’t stand much of a chance. The fake and real clips were set to the same resolution and length, and the volunteers had 10 seconds to decide which showed real people in the real world and which were made by AI (reflecting the fast pace of social media scrolling).
The test was commissioned to mark the wider rollout of Runway’s new Gen-4.5 model, which the company promises offers “unprecedented visual fidelity and creative control”, as well as results that are “cinematic and highly realistic”.
‘Has a more critical mindset’
Look at
Participants involved in the test performed best when human faces, hands or actions were involved, with accuracy rates between 58-65% for these clips. Germanidis says this may be because these videos are where ‘spooky’ elements are most noticeable, although the AI videos themselves are still of the same quality as the real ones.
Germanidis urges people to “have a more critical mindset” when weighing everything they see online. “At this point we are crossing the threshold and it is difficult to distinguish generated from real videos,” he told The Information.
Runway is working on ways to reliably watermark AI output, where the videos already contain metadata identifying them as AI-generated by default. It’s something that governments are keen to see implemented, understandably so, as the line between the real and the computer-generated disappears completely.
However, Runway Gen-4.5 is not perfect yet. Runway says it can still struggle with disappearing objects, causal reasoning (so doors can open before the handle is pulled) and success bias (so poorly aimed kicks still fly into the net in football scenes, for example).
Follow TechRadar on Google News and add us as a preferred source to get our expert news, reviews and opinions in your feeds. Be sure to click the Follow button!
And of course you can too follow TechRadar on TikTok for news, reviews, video unboxings, and get regular updates from us on WhatsApp also.
The best business laptops for all budgets
