# Assessing the Potential of Generative Agents in Crowdsourced Fact-Checking

arXiv: https://arxiv.org/abs/2504.19940 · HTML: https://mlsanity.com/paper/2504.19940/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Placed in Web, search, and research agents, one of 16 directions in LLM agents and tool use.
- Builds on 32 of its 67 references that are in the map.
- First posted on arXiv 24 Apr 2025.

## Placed in

- LLM agents and tool use / [Web, search, and research agents](https://mlsanity.com/agents/web-search-and-research-agents/)

## Builds on (most cited first, 30 of 32)

- [Extracting Training Data from Large Language Models](https://mlsanity.com/paper/2012.07805/) (2020)
- [Survey of Hallucination in Natural Language Generation](https://mlsanity.com/paper/2202.03629/) (2022)
- [AutoGen: Enabling Next-Gen LLM Applications via Multi-Agent Conversation](https://mlsanity.com/paper/2308.08155/) (2023)
- [Self-RAG: Learning to Retrieve, Generate, and Critique through Self-Reflection](https://mlsanity.com/paper/2310.11511/) (2023)
- [BERTopic: Neural topic modeling with a class-based TF-IDF procedure](https://mlsanity.com/paper/2203.05794/) (2022)
- [A Survey on Automated Fact-Checking](https://mlsanity.com/paper/2108.11896/) (2021)
- [Benchmarking Large Language Models in Retrieval-Augmented Generation](https://mlsanity.com/paper/2309.01431/) (2023)
- [MiniCheck: Efficient Fact-Checking of LLMs on Grounding Documents](https://mlsanity.com/paper/2404.10774/) (2024)
- [S3: Social-network Simulation System with Large Language Model-Empowered Agents](https://mlsanity.com/paper/2307.14984/) (2023)
- [Fact-Checking Complex Claims with Program-Guided Reasoning](https://mlsanity.com/paper/2305.12744/) (2023)
- [Towards LLM-based Fact Verification on News Claims with a Hierarchical Step-by-Step Prompting Method](https://mlsanity.com/paper/2310.00305/) (2023)
- [Automated Fact Checking: Task formulations, methods and future directions](https://mlsanity.com/paper/1806.07687/) (2018)
- [Moral Alignment for LLM Agents](https://mlsanity.com/paper/2410.01639/) (2024)
- [FACT-GPT: Fact-Checking Augmentation via Claim Matching with LLMs](https://mlsanity.com/paper/2402.05904/) (2024)
- [Agent-Based Modelling Meets Generative AI in Social Network Simulations](https://mlsanity.com/paper/2411.16031/) (2024)
- [Birdwatch: Crowd Wisdom and Bridging Algorithms can Inform Understanding and Reduce the Spread of Misinformation](https://mlsanity.com/paper/2210.15723/) (2022)
- [Generative Agent-Based Modeling: Unveiling Social System Dynamics through Coupling Mechanistic Models with Generative Artificial Intelligence](https://mlsanity.com/paper/2309.11456/) (2023)
- [Generative Large Language Models in Automated Fact-Checking: A Survey](https://mlsanity.com/paper/2407.02351/) (2024)
- [Crowdsourced Fact-Checking at Twitter: How Does the Crowd Compare With Experts?](https://mlsanity.com/paper/2208.09214/) (2022)
- [Reinforcement Retrieval Leveraging Fine-grained Feedback for Fact Checking News Claims with Black-Box LLM](https://mlsanity.com/paper/2404.17283/) (2024)
- [Community-Based Fact-Checking on Twitter's Birdwatch Platform](https://mlsanity.com/paper/2104.07175/) (2021)
- [Large Language Model Agent for Fake News Detection](https://mlsanity.com/paper/2405.01593/) (2024)
- [Are LLMs Socially Adaptive? Contrasting Belief Evolution in Large Language Models and Humans](https://mlsanity.com/paper/2410.10398/) (2024)
- [Can Generative Agent-Based Modeling Replicate the Friendship Paradox in Social Media Simulations?](https://mlsanity.com/paper/2502.05919/) (2025)
- [Can The Crowd Identify Misinformation Objectively? The Effects of Judgment Scale and Assessor's Background](https://mlsanity.com/paper/2005.06915/) (2020)
- [Claim Check-Worthiness Detection: How Well do LLMs Grasp Annotation Guidelines?](https://mlsanity.com/paper/2404.12174/) (2024)
- [A Survey on the Role of Crowds in Combating Online Misinformation: Annotators, Evaluators, and Creators](https://mlsanity.com/paper/2310.02095/) (2023)
- [The Many Dimensions of Truthfulness: Crowdsourcing Misinformation Assessments on a Multidimensional Scale](https://mlsanity.com/paper/2108.01222/) (2021)
- [Towards Automated Fact-Checking of Real-World Claims: Exploring Task Formulation and Assessment with LLMs](https://mlsanity.com/paper/2502.08909/) (2025)
- [Community Notes vs. Snoping: How the Crowd Selects Fact-Checking Targets on Social Media](https://mlsanity.com/paper/2305.09519/) (2023)

## Built on by (most cited first, 3 of 3)

- [Community Moderation and the New Epistemology of Fact Checking on Social Media](https://mlsanity.com/paper/2505.20067/) (2025)
- [CrowdLLM: Building LLM-Based Digital Populations Augmented with Generative Models](https://mlsanity.com/paper/2512.07890/) (2025)
- [Large language models for spreading dynamics in complex systems](https://mlsanity.com/paper/2602.08085/) (2026)

## Abstract (arXiv)

The growing spread of online misinformation has created an urgent need for scalable, reliable fact-checking solutions. Crowdsourced fact-checking - where non-experts evaluate claim veracity - offers a cost-effective alternative to expert verification, despite concerns about variability in quality and bias. Encouraged by promising results in certain contexts, major platforms such as X (formerly Twitter), Facebook, and Instagram have begun shifting from centralized moderation to decentralized, crowd-based approaches. In parallel, advances in Large Language Models (LLMs) have shown strong performance across core fact-checking tasks, including claim detection and evidence evaluation. However, their potential role in crowdsourced workflows remains unexplored. This paper investigates whether LLM-powered generative agents - autonomous entities that emulate human behavior and decision-making - can meaningfully contribute to fact-checking tasks traditionally reserved for human crowds. Using the protocol of La Barbera et al. (2024), we simulate crowds of generative agents with diverse demographic and ideological profiles. Agents retrieve evidence, assess claims along multiple quality dimensions, and issue final veracity judgments. Our results show that agent crowds outperform human crowds in truthfulness classification, exhibit higher internal consistency, and show reduced susceptibility to social and cognitive biases. Compared to humans, agents rely more systematically on informative criteria such as Accuracy, Precision, and Informativeness, suggesting a more structured decision-making process. Overall, our findings highlight the potential of generative agents as scalable, consistent, and less biased contributors to crowd-based fact-checking systems.