After every dialog, individuals had been requested the identical ranking questions. The researchers adopted up with all of the individuals 10 days after the experiment, after which two months later, to assess whether or not their views had modified following the dialog with the AI bot. The individuals reported a 20% discount of perception in their chosen conspiracy principle on common, suggesting that speaking to the bot had basically modified some people’s minds.
“Even in a lab setting, 20% is a large effect on changing people’s beliefs,” says Zhang. “It might be weaker in the real world, but even 10% or 5% would still be very substantial.”
The authors sought to safeguard in opposition to AI fashions’ tendency to make up data—often called hallucinating—by using an expert fact-checker to consider the accuracy of 128 claims the AI had made. Of these, 99.2% had been discovered to be true, whereas 0.8% had been deemed deceptive. None had been discovered to be fully false.
One rationalization for this excessive diploma of accuracy is that so much has been written about conspiracy theories on the web, making them very nicely represented in the mannequin’s coaching knowledge, says David G. Rand, a professor at MIT Sloan who additionally labored on the venture. The adaptable nature of GPT-4 Turbo means it may simply be related to totally different platforms for customers to work together with in the longer term, he provides.
“You could imagine just going to conspiracy forums and inviting people to do their own research by debating the chatbot,” he says. “Similarly, social media could be hooked up to LLMs to post corrective responses to people sharing conspiracy theories, or we could buy Google search ads against conspiracy-related search terms like ‘Deep State.’”
The analysis upended the authors’ preconceived notions about how receptive people had been to strong proof debunking not solely conspiracy theories, but additionally different beliefs that aren’t rooted in good-quality data, says Gordon Pennycook, an affiliate professor at Cornell University who additionally labored on the venture.
“People were remarkably responsive to evidence. And that’s really important,” he says. “Evidence does matter.”