Study Finds LLM Narrative Explanations Make People Trust AI More — Even When It's Wrong
Published · updated · curated by AI Is Going Just Great
Source: arxiv.org ↗
More persuasive narratives may have had a detrimental effect on decision response times and the ability to discriminate between a correct and incorrect AI prediction.
A large-scale behavioral experiment found that when LLMs provide persuasive, story-like explanations for their predictions, people don't actually make better decisions — they just rely on the AI more, regardless of whether it's correct. In other words, a more compelling AI story increases your willingness to follow it off a cliff.
The researchers also found that more persuasive narratives may have slowed response times and made it harder for people to distinguish a correct AI prediction from an incorrect one. So the better the AI is at explaining itself, the worse humans may become at catching its mistakes. Explainable AI, it turns out, might be most persuasive precisely when it needs the most scrutiny.