The Thin Line Between Conspiracy Theories and Opinion: Why Humans and AI Struggle to Differentiate Them
Abstract
This study addresses the challenges in identifying online conspiracy theories, both by humans and automated systems. It relies on a corpus comprising conspiracy, news, and opinion articles gathered from the Portuguese blogosphere. Each article underwent evaluation through (i) InfoRadar, a multidimensional article characterization tool; (ii) ChatGPT 3.5, which involved an in-depth content analysis for key classification features; and (iii) a survey in which a group of online readers assessed various indicators for article categorization and credibility assessment. The mixed-methods approach highlights the difficulties faced by both humans and machines in differentiating conspiracy from opinion articles and provides valuable insights for distinguishing these categories. This research not only enhances our understanding of credibility perception in content marked by information disorder but also offers insights for developing transparent and explainable tools for critically assessing conspiracy theories.