mlsanityData to 14 Oct 2026

Assessing the Potential of Generative Agents in Crowdsourced Fact-Checking

arXiv 2504.19940 ↗First posted 24 Apr 2025cs.CL cs.AI cs.MAcited by 3

On the map: LLM agents and tool use → Web, search, and research agents

Abstract (arXiv)

The growing spread of online misinformation has created an urgent need for scalable, reliable fact-checking solutions. Crowdsourced fact-checking - where non-experts evaluate claim veracity - offers a cost-effective alternative to expert verification, despite concerns about variability in quality and bias. Encouraged by promising results in certain contexts, major platforms such as X (formerly Twitter), Facebook, and Instagram have begun shifting from centralized moderation to decentralized, crowd-based approaches. In parallel, advances in Large Language Models (LLMs) have shown strong performance across core fact-checking tasks, including claim detection and evidence evaluation. However, their potential role in crowdsourced workflows remains unexplored. This paper investigates whether LLM-powered generative agents - autonomous entities that emulate human behavior and decision-making - can meaningfully contribute to fact-checking tasks traditionally reserved for human crowds. Using the protocol of La Barbera et al. (2024), we simulate crowds of generative agents with diverse demographic and ideological profiles. Agents retrieve evidence, assess claims along multiple quality dimensions, and issue final veracity judgments. Our results show that agent crowds outperform human crowds in truthfulness classification, exhibit higher internal consistency, and show reduced susceptibility to social and cognitive biases. Compared to humans, agents rely more systematically on informative criteria such as Accuracy, Precision, and Informativeness, suggesting a more structured decision-making process. Overall, our findings highlight the potential of generative agents as scalable, consistent, and less biased contributors to crowd-based fact-checking systems.

Key facts. Cited by 3 papers in the map (data through 14 Oct 2026). Placed in Web, search, and research agents, one of 16 directions in LLM agents and tool use. Builds on 32 of its 67 references that are in the map. First posted on arXiv 24 Apr 2025.

Builds on

32 of 67 references.

Closest ancestors · 1

Same direction, newest first

  1. 2024Large Language Model Agent for Fake News Detection30 Apr

Other references · 31

  1. 2020Extracting Training Data from Large Language Modelscited by 2,189
  2. 2022Survey of Hallucination in Natural Language Generationcited by 2,075
  3. 2023AutoGen: Enabling Next-Gen LLM Applications via Multi-Agent Conversationcited by 1,988
  4. 2023Self-RAG: Learning to Retrieve, Generate, and Critique through Self-Reflectioncited by 1,839
  5. 2022BERTopic: Neural topic modeling with a class-based TF-IDF procedurecited by 801
  6. 2021A Survey on Automated Fact-Checkingcited by 378
  7. 2023Benchmarking Large Language Models in Retrieval-Augmented Generationcited by 312
  8. 2024MiniCheck: Efficient Fact-Checking of LLMs on Grounding Documentscited by 257
  9. 2023S3: Social-network Simulation System with Large Language Model-Empowered Agentscited by 233
  10. 2023Fact-Checking Complex Claims with Program-Guided Reasoningcited by 146
  11. 2023Towards LLM-based Fact Verification on News Claims with a Hierarchical Step-by-Step Prompting Methodcited by 97
  12. 2018Automated Fact Checking: Task formulations, methods and future directionscited by 96
19 more
  1. 2024Moral Alignment for LLM Agentscited by 39
  2. 2024FACT-GPT: Fact-Checking Augmentation via Claim Matching with LLMscited by 27
  3. 2024Agent-Based Modelling Meets Generative AI in Social Network Simulationscited by 26
  4. 2022Birdwatch: Crowd Wisdom and Bridging Algorithms can Inform Understanding and Reduce the Spread of Misinformationcited by 21
  5. 2023Generative Agent-Based Modeling: Unveiling Social System Dynamics through Coupling Mechanistic Models with Generative Artificial Intelligencecited by 19
  6. 2024Generative Large Language Models in Automated Fact-Checking: A Surveycited by 19
  7. 2022Crowdsourced Fact-Checking at Twitter: How Does the Crowd Compare With Experts?cited by 16
  8. 2024Reinforcement Retrieval Leveraging Fine-grained Feedback for Fact Checking News Claims with Black-Box LLMcited by 15
  9. 2021Community-Based Fact-Checking on Twitter's Birdwatch Platformcited by 13
  10. 2024Are LLMs Socially Adaptive? Contrasting Belief Evolution in Large Language Models and Humanscited by 11
  11. 2025Can Generative Agent-Based Modeling Replicate the Friendship Paradox in Social Media Simulations?cited by 10
  12. 2020Can The Crowd Identify Misinformation Objectively? The Effects of Judgment Scale and Assessor's Backgroundcited by 9
  13. 2024Claim Check-Worthiness Detection: How Well do LLMs Grasp Annotation Guidelines?cited by 9
  14. 2023A Survey on the Role of Crowds in Combating Online Misinformation: Annotators, Evaluators, and Creatorscited by 6
  15. 2021The Many Dimensions of Truthfulness: Crowdsourcing Misinformation Assessments on a Multidimensional Scalecited by 5
  16. 2025Towards Automated Fact-Checking of Real-World Claims: Exploring Task Formulation and Assessment with LLMscited by 5
  17. 2023Community Notes vs. Snoping: How the Crowd Selects Fact-Checking Targets on Social Mediacited by 3
  18. 2024The Real, the Better: Aligning Large Language Models with Online Human Behaviorscited by 3
  19. 2023Evidence-based Interpretable Open-domain Fact-checking with Large Language Modelscited by 2

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 2; 2026: 1.

3 papers.

  1. 2025Community Moderation and the New Epistemology of Fact Checking on Social Mediacited by 5
  2. 2025CrowdLLM: Building LLM-Based Digital Populations Augmented with Generative Modelscited by 5
  3. 2026Large language models for spreading dynamics in complex systemscited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2023The Perils & Promises of Fact-checking with Large Language Models
  2. 2025Fact-checking with Generative AI: A Systematic Cross-Topic Examination of LLMs Capacity to Detect Veracity of Political Information
  3. 2024Can LLMs Produce Faithful Explanations For Fact-checking? Towards Faithful Explainable Fact-Checking via Multi-Agent Debate
  4. 2026Leveraging LLM Parametric Knowledge for Fact Checking without Retrieval
  5. 2025Towards Robust Fact-Checking: A Multi-Agent System with Advanced Evidence Retrieval
  6. 2026TRUST Agents: A Collaborative Multi-Agent Framework for Fake News Detection, Explainable Verification, and Logic-Aware Claim Reasoning
  7. 2025Large Language Models Require Curated Context for Reliable Political Fact-Checking -- Even with Reasoning and Web Search
  8. 2025Toward Verifiable Misinformation Detection: A Multi-Tool LLM Agent Framework