mlsanityData to 13 Oct 2026

COMMUNITY-CROSS-INSTRUCT: Unsupervised Instruction Generation for Aligning Large Language Models to Online Communities

arXiv 2406.12074 ↗First posted 17 Jun 2024cs.CLcited by 12

On the map: LLM alignment / RLHF / preference optimisation → Human values and AI feedback alignment

Abstract (arXiv)

Social scientists use surveys to probe the opinions and beliefs of populations, but these methods are slow, costly, and prone to biases. Recent advances in large language models (LLMs) enable the creating of computational representations or "digital twins" of populations that generate human-like responses mimicking the population's language, styles, and attitudes. We introduce Community-Cross-Instruct, an unsupervised framework for aligning LLMs to online communities to elicit their beliefs. Given a corpus of a community's online discussions, Community-Cross-Instruct automatically generates instruction-output pairs by an advanced LLM to (1) finetune a foundational LLM to faithfully represent that community, and (2) evaluate the alignment of the finetuned model to the community. We demonstrate the method's utility in accurately representing political and diet communities on Reddit. Unlike prior methods requiring human-authored instructions, Community-Cross-Instruct generates instructions in a fully unsupervised manner, enhancing scalability and generalization across domains. This work enables cost-effective and automated surveying of diverse online communities.

Key facts. Cited by 12 papers in the map (data through 13 Oct 2026). Placed in Human values and AI feedback alignment, one of 10 directions in LLM alignment / RLHF / preference optimisation. Builds on 21 of its 43 references that are in the map. First posted on arXiv 17 Jun 2024.

Builds on

21 of 43 references.

References · 21

  1. 2022Training language models to follow instructions with human feedbackcited by 15,856
  2. 2021LoRA: Low-Rank Adaptation of Large Language Modelscited by 14,231
  3. 2022Self-Instruct: Aligning Language Models with Self-Generated Instructionscited by 2,693
  4. 2020MPNet: Masked and Permuted Pre-training for Language Understandingcited by 976
  5. 2022BERTopic: Neural topic modeling with a class-based TF-IDF procedurecited by 792
  6. 2023Whose Opinions Do Language Models Reflect?cited by 768
  7. 2022Out of One, Many: Using Language Models to Simulate Human Samplescited by 747
  8. 2023Instruction Tuning for Large Language Models: A Surveycited by 622
  9. 2024Self-Play Fine-Tuning Converts Weak Language Models to Strong Language Modelscited by 552
  10. 2023Towards Measuring the Representation of Subjective Global Opinions in Language Modelscited by 428
  11. 2024The Butterfly Effect of Altering Prompts: How Small Changes and Jailbreaks Affect Large Language Model Performancecited by 98
  12. 2023Language Models Trained on Media Diets Can Predict Public Opinioncited by 30
9 more
  1. 2024How Susceptible are Large Language Models to Ideological Manipulation?cited by 26
  2. 2024Human-Instruction-Free LLM Self-Alignment with Limited Samplescited by 25
  3. 2024Whose Emotions and Moral Sentiments Do Language Models Reflect?cited by 20
  4. 2023Large Language Models as Subpopulation Representative Models: A Reviewcited by 16
  5. 2024Harmful Speech Detection by Language Models Exhibits Gender-Queer Dialect Biascited by 16
  6. 2024Reading Between the Tweets: Deciphering Ideological Stances of Interconnected Mixed-Ideology Communitiescited by 7
  7. 2024IsamasRed: A Public Dataset Tracking Reddit Discussions on Israel-Hamas Conflictcited by 6
  8. 2023Non-Binary Gender Expression in Online Interactionscited by 5
  9. 2022Polling Latent Opinions: A Method for Computational Sociolinguistics Using Transformer Language Modelscited by 1

Built on by

Cited by year (2026 to 13 Oct)

Citing papers per year: 2024: 4; 2025: 7; 2026: 1.

12 papers.

  1. 2025Language Model Fine-Tuning on Scaled Survey Data for Predicting Distributions of Public Opinionscited by 65
  2. 2024Y Social: an LLM-powered Social Media Digital Twincited by 43
  3. 2025STEER-BENCH: A Benchmark for Evaluating the Steerability of Large Language Modelscited by 7
  4. 2024Leveraging Machine Learning to Identify Gendered Stereotypes and Body Image Concerns on Diet and Fitness Online Forumscited by 3
  5. 2024SubData: Bridging Heterogeneous Datasets to Enable Theory-Driven Evaluation of Political and Demographic Perspectives in LLMscited by 3
  6. 2025MDIT: A Model-free Data Interpolation Method for Diverse Instruction Tuningcited by 3
  7. 2024Improving and Assessing the Fidelity of Large Language Models Alignment to Online Communitiescited by 2
  8. 2025AlignSurvey: A Comprehensive Benchmark for Human Preferences Alignment in Social Surveyscited by 2
  9. 2025Synthia: Scalable Grounded Persona Generation from Social Media Datacited by 1
  10. 2025Graph-Based Alternatives to LLMs for Human Simulationcited by 1
  11. 2025Community-Aligned Behavior Under Uncertainty: Evidence of Epistemic Stance Transfer in LLMscited by 0
  12. 2026Density-Guided Response Optimization: Community-Grounded Alignment via Implicit Acceptance Signalscited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2026Modeling Community Attitude through Reaction Tone: A Human-AI Collaborative Framework for Evaluating LLM Alignment with Linguistic Behaviors in Online Communities
  2. 2026CommunityBench: Benchmarking Community-Level Alignment across Diverse Groups and Tasks
  3. 2026VASTU: Language Models Struggle to Recognize Online Community Values
  4. 2026The Limits of Simulated Societies: How Post-Training and Survey Fine-Tuning Erase Cross-Cultural Variance
  5. 2025Too Open for Opinion? Embracing Open-Endedness in Large Language Models for Social Simulation
  6. 2026Mapping the Emerging Social Science of Large Language Models
  7. 2024Benchmarking Distributional Alignment of Large Language Models
  8. 2026PluRule: A Benchmark for Moderating Pluralistic Communities on Social Media