mlsanityData to 7 Oct 2026

CommunityBench: Benchmarking Community-Level Alignment across Diverse Groups and Tasks

arXiv 2601.13669 ↗First posted 20 Jan 2026cs.CLcited by 1

On the map: LLM alignment / RLHF / preference optimisation → Human values and AI feedback alignment

Abstract (arXiv)

Large language models (LLMs) alignment ensures model behaviors reflect human value. Existing alignment strategies primarily follow two paths: one assumes a universal value set for a unified goal (i.e., one-size-fits-all), while the other treats every individual as unique to customize models (i.e., individual-level). However, assuming a monolithic value space marginalizes minority norms, while tailoring individual models is prohibitively expensive. Recognizing that human society is organized into social clusters with high intra-group value alignment, we propose community-level alignment as a "middle ground". Practically, we introduce CommunityBench, the first large-scale benchmark for community-level alignment evaluation, featuring four tasks grounded in Common Identity and Common Bond theory. With CommunityBench, we conduct a comprehensive evaluation of various foundation models on CommunityBench, revealing that current LLMs exhibit limited capacity to model community-specific preferences. Furthermore, we investigate the potential of community-level alignment in facilitating individual modeling, providing a promising direction for scalable and pluralistic alignment.

Key facts. Cited by 1 paper in the map (data through 7 Oct 2026). Placed in Human values and AI feedback alignment, one of 10 directions in LLM alignment / RLHF / preference optimisation. Builds on 24 of its 43 references that are in the map. First posted on arXiv 20 Jan 2026.

Builds on

24 of 43 references.

Closest ancestors · 7

Same direction, newest first

  1. 2025A Survey on Personalized Alignment -- The Missing Piece for Large Language Models in Real-World Applications21 Mar
  2. 2025From 1,000,000 Users to Every User: Scaling Up Personalized Preference for User-level Alignment19 Mar
  3. 2024Benchmarking Distributional Alignment of Large Language Models8 Nov
  4. 2024ComPO: Community Preferences for Language Model Personalization21 Oct
  5. 2024Aligning LLMs with Individual Preferences via Interaction4 Oct
  6. 2024Modular Pluralism: Pluralistic Alignment via Multi-LLM Collaboration22 Jun
  7. 2023The Past, Present and Better Future of Feedback Learning in Large Language Models for Subjective Human Preferences and Values11 Oct

Other references · 17

  1. 2022Training language models to follow instructions with human feedbackcited by 15,712
  2. 2023Direct Preference Optimization: Your Language Model is Secretly a Reward Modelcited by 7,916
  3. 2021Finetuned Language Models Are Zero-Shot Learnerscited by 3,747
  4. 2024ChatGLM: A Family of Large Language Models from GLM-130B to GLM-4 All Toolscited by 1,096
  5. 2021Ethical and social risks of harm from Language Modelscited by 957
  6. 2023Whose Opinions Do Language Models Reflect?cited by 759
  7. 2024ORPO: Monolithic Preference Optimization without Reference Modelcited by 556
  8. 2022Improving alignment of dialogue agents via targeted human judgementscited by 527
  9. 2020Artificial Intelligence, Values and Alignmentcited by 443
  10. 2018Scalable agent alignment via reward modeling: a research directioncited by 438
  11. 2023Distributional Preference Learning: Understanding and Accounting for Hidden Context in RLHFcited by 125
  12. 2023SteerLM: Attribute Conditioned SFT as an (User-Steerable) Alternative to RLHFcited by 107
5 more
  1. 2024Human Alignment of Large Language Models through Online Preference Optimisationcited by 90
  2. 2023DICES Dataset: Diversity in Conversational AI Evaluation for Safetycited by 86
  3. 2023Everyone Deserves A Reward: Learning Customized Human Preferencescited by 42
  4. 2023CDEval: A Benchmark for Measuring the Cultural Dimensions of Large Language Modelscited by 23
  5. 2024Skin-in-the-Game: Decision Making via Multi-Stakeholder Alignment in LLMscited by 8

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2026: 1.

1 paper.

  1. 2026Modeling Community Attitude through Reaction Tone: A Human-AI Collaborative Framework for Evaluating LLM Alignment with Linguistic Behaviors in Online Communitiescited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025Cultivating Pluralism In Algorithmic Monoculture: The Community Alignment Dataset
  2. 2026Density-Guided Response Optimization: Community-Grounded Alignment via Implicit Acceptance Signals
  3. 2023From Instructions to Intrinsic Human Values -- A Survey of Alignment Goals for Big Models
  4. 2026Alignment by Stereotyping: How LLMs Sacrifice Individual Distinctiveness for Cultural Adaptation
  5. 2024COMMUNITY-CROSS-INSTRUCT: Unsupervised Instruction Generation for Aligning Large Language Models to Online Communities
  6. 2025Operationalizing Pluralistic Values in Large Language Model Alignment Reveals Trade-offs in Safety, Inclusivity, and Model Behavior
  7. 2024Improving and Assessing the Fidelity of Large Language Models Alignment to Online Communities
  8. 2023Heterogeneous Value Alignment Evaluation for Large Language Models