The Explanation Trap: When AI Advice Backfires

Physicians’ pre-existing accuracy ironically diminishes trust in artificial intelligence; the more often a doctor is correct, the less likely others are to accept an AI’s differing diagnosis, highlighting how established authority creates a bias against even demonstrably superior algorithmic reasoning, a phenomenon rooted in the human tendency to prioritize familiar narratives over objective data, even when those narratives are flawed-a cognitive shortcut quantified as $P(trust) = f(prior \ correctness, AI \ disagreement)$.

New research reveals that providing explanations for artificial intelligence recommendations in medical diagnoses doesn’t always improve decision-making, and can actually decrease accuracy when the AI is wrong.