# How Human Feedback Shapes AI-generated Community Notes

arXiv: https://arxiv.org/abs/2606.30905 · HTML: https://mlsanity.com/paper/2606.30905/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Placed in Human values and AI feedback alignment, one of 10 directions in LLM alignment / RLHF / preference optimisation.
- Builds on 15 of its 39 references that are in the map.
- First posted on arXiv 29 Jun 2026.

## Placed in

- LLM alignment / RLHF / preference optimisation / [Human values and AI feedback alignment](https://mlsanity.com/alignment/human-values-and-ai-feedback-alignment/)

## Builds on (most cited first, 15 of 15)

- [Beyond English-Centric Multilingual Machine Translation](https://mlsanity.com/paper/2010.11125/) (2020)
- [FACT-GPT: Fact-Checking Augmentation via Claim Matching with LLMs](https://mlsanity.com/paper/2402.05904/) (2024)
- [Supernotes: Driving Consensus in Crowd-Sourced Fact-Checking](https://mlsanity.com/paper/2411.06116/) (2024)
- [Birdwatch: Crowd Wisdom and Bridging Algorithms can Inform Understanding and Reduce the Spread of Misinformation](https://mlsanity.com/paper/2210.15723/) (2022)
- [Community notes reduce engagement with and diffusion of false information online](https://mlsanity.com/paper/2502.13322/) (2025)
- [Beyond the Crowd: LLM-Augmented Community Notes for Governing Health Misinformation](https://mlsanity.com/paper/2510.11423/) (2025)
- [Scaling Human Judgment in Community Notes with LLMs](https://mlsanity.com/paper/2506.24118/) (2025)
- [Community Notes vs. Snoping: How the Crowd Selects Fact-Checking Targets on Social Media](https://mlsanity.com/paper/2305.09519/) (2023)
- [Fact-checking with Generative AI: A Systematic Cross-Topic Examination of LLMs Capacity to Detect Veracity of Political Information](https://mlsanity.com/paper/2503.08404/) (2025)
- [Grok in the Wild: Characterizing the Roles and Uses of Large Language Models on Social Media](https://mlsanity.com/paper/2602.11286/) (2026)
- [AI Fact-Checking in the Wild: A Field Evaluation of LLM-Written Community Notes on X](https://mlsanity.com/paper/2604.02592/) (2026)
- [Searching For or Reviewing Evidence Improves Crowdworkers' Misinformation Judgments and Reduces Partisan Bias](https://mlsanity.com/paper/2108.07898/) (2021)
- [Community Notes are Vulnerable to Rater Bias and Manipulation](https://mlsanity.com/paper/2511.02615/) (2025)
- [The Laziness of the Crowd: Effort Aversion Among Raters Risks Undermining the Efficacy of X's Community Notes Program](https://mlsanity.com/paper/2603.11120/) (2026)
- [Quality-Sensitive Matrix Factorization for Community Notes: Towards Sample Efficiency and Manipulation Resistance](https://mlsanity.com/paper/2604.11224/) (2026)

## Built on by (most cited first, 1 of 1)

- [Whitewashing Hate, Smearing Harmless Content: Annotator-Style Rebuttal Attacks on LLM-Based Moderation](https://mlsanity.com/paper/2608.22230/) (2026)

## Abstract (arXiv)

Community Notes, a bridging-based crowd-sourced fact-checking system, has emerged as a new mechanism for moderating misleading information on social media and has been adopted by major platforms including X, Facebook, Instagram, Threads, and TikTok. Since its introduction, there has been an open question about what role AI could play in scaling and optimizing the system. Recently, X extended its Community Notes system by introducing Collaborative Notes: notes initially drafted by an LLM and iteratively refined based on feedback from human contributors. In this work, we systematically analyze the complete corpus of 19,146 collaborative notes and 211,850 instances of human feedback. First, we develop a taxonomy of human suggestions for improving AI-generated note drafts and find that suggestions involving factual corrections and additional context are most likely to be incorporated, while subjective policy judgments rarely are. Second, we examine changes in helpfulness across versions of collaborative notes and find that human feedback leads to more helpful notes, with the greatest impact coming from suggestions that challenge the main claim in the previous draft, particularly when submitted by more active contributors. Finally, we find that although collaborative notes improve through human feedback, they reach helpful status and are shown on the platform at lower rates than human-only or AI-only notes, with limited human participation emerging as a key bottleneck. Nevertheless, rather than serving as a weaker substitute, collaborative notes tend to play a complementary role, predominantly targeting posts that do not attract human-only or AI-only notes. Our analysis provides an initial description of efforts to use AI to improve crowdsourced content moderation in a real-world moderation system and outlines pathways for future improvements to such features.