# Supernotes: Driving Consensus in Crowd-Sourced Fact-Checking

arXiv: https://arxiv.org/abs/2411.06116 · HTML: https://mlsanity.com/paper/2411.06116/

## Key facts

- Cited by 22 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 9 Nov 2024.

## Built on by (most cited first, 22 of 22)

- [Value Profiles for Encoding Human Variation](https://mlsanity.com/paper/2503.15484/) (2025)
- [Generative Social Choice: The Next Generation](https://mlsanity.com/paper/2505.22939/) (2025)
- [Beyond the Crowd: LLM-Augmented Community Notes for Governing Health Misinformation](https://mlsanity.com/paper/2510.11423/) (2025)
- [Can Language Models Falsify? Evaluating Algorithmic Reasoning with Counterexample Creation](https://mlsanity.com/paper/2502.19414/) (2025)
- [Community Moderation and the New Epistemology of Fact Checking on Social Media](https://mlsanity.com/paper/2505.20067/) (2025)
- [COMMUNITYNOTES: A Dataset for Exploring the Helpfulness of Fact-Checking Explanations](https://mlsanity.com/paper/2510.24810/) (2025)
- [LLMs as mediators: Can they diagnose conflicts accurately?](https://mlsanity.com/paper/2412.14675/) (2024)
- [Question the Questions: Auditing Representation in Online Deliberative Processes](https://mlsanity.com/paper/2511.04588/) (2025)
- [Large Language Models Require Curated Context for Reliable Political Fact-Checking -- Even with Reasoning and Web Search](https://mlsanity.com/paper/2511.18749/) (2025)
- [GitSearch: Enhancing Community Notes Generation with Gap-Informed Targeted Search](https://mlsanity.com/paper/2602.08945/) (2026)
- [The Synthetic Media Shift: Tracking the Rise, Virality, and Detectability of AI-Generated Multimodal Misinformation](https://mlsanity.com/paper/2604.15372/) (2026)
- [Finding Common Ground in a Sea of Alternatives](https://mlsanity.com/paper/2603.16751/) (2026)
- [Towards Proactive Information Probing: Customer Service Chatbots Harvesting Value from Conversation](https://mlsanity.com/paper/2604.11077/) (2026)
- [How Human Feedback Shapes AI-generated Community Notes](https://mlsanity.com/paper/2606.30905/) (2026)
- [Can AI mediation improve democratic deliberation?](https://mlsanity.com/paper/2601.05904/) (2026)
- [XNote: Benchmarking Automated Community Notes Generation for Image-based Contextual Deception](https://mlsanity.com/paper/2603.22453/) (2026)
- [Better with Experience: Self-Evolving LLM Agents for Evidence-Grounded Health Community Notes](https://mlsanity.com/paper/2606.02215/) (2026)
- [Towards Multi-Agent-Simulation-Based Community Note Evaluation](https://mlsanity.com/paper/2606.18268/) (2026)
- [Gaming Consensus: Coordinated Manipulation in Crowdsourced Fact-Checking](https://mlsanity.com/paper/2607.01824/) (2026)
- [AHEAD: Advancing Multi-Class Label Aggregation with Interpretable Cross-Annotator Modeling](https://mlsanity.com/paper/2607.18465/) (2026)
- [CoVer: Conflict-Aware Claim Verification](https://mlsanity.com/paper/2609.00508/) (2026)
- [Community-Driven API and AI Writer Design for Openly Scaling Community Notes](https://mlsanity.com/paper/2609.40067/) (2026)

## Abstract (arXiv)

X's Community Notes, a crowd-sourced fact-checking system, allows users to annotate potentially misleading posts. Notes rated as helpful by a diverse set of users are prominently displayed below the original post. While demonstrably effective at reducing misinformation's impact when notes are displayed, there is an opportunity for notes to appear on many more posts: for 91% of posts where at least one note is proposed, no notes ultimately achieve sufficient support from diverse users to be shown on the platform. This motivates the development of Supernotes: AI-generated notes that synthesize information from several existing community notes and are written to foster consensus among a diverse set of users. Our framework uses an LLM to generate many diverse Supernote candidates from existing proposed notes. These candidates are then evaluated by a novel scoring model, trained on millions of historical Community Notes ratings, selecting candidates that are most likely to be rated helpful by a diverse set of users. To test our framework, we ran a human subjects experiment in which we asked participants to compare the Supernotes generated by our framework to the best existing community notes for 100 sample posts. We found that participants rated the Supernotes as significantly more helpful, and when asked to choose between the two, preferred the Supernotes 75.2% of the time. Participants also rated the Supernotes more favorably than the best existing notes on quality, clarity, coverage, context, and argumentativeness. Finally, in a follow-up experiment, we asked participants to compare the Supernotes against LLM-generated summaries and found that the participants rated the Supernotes significantly more helpful, demonstrating that both the LLM-based candidate generation and the consensus-driven scoring play crucial roles in creating notes that effectively build consensus among diverse users.