# COMMUNITY-CROSS-INSTRUCT: Unsupervised Instruction Generation for Aligning Large Language Models to Online Communities

arXiv: https://arxiv.org/abs/2406.12074 · HTML: https://mlsanity.com/paper/2406.12074/

## Key facts

- Cited by 12 papers in the map (data through 14 Oct 2026).
- Placed in Human values and AI feedback alignment, one of 10 directions in LLM alignment / RLHF / preference optimisation.
- Builds on 21 of its 43 references that are in the map.
- First posted on arXiv 17 Jun 2024.

## Placed in

- LLM alignment / RLHF / preference optimisation / [Human values and AI feedback alignment](https://mlsanity.com/alignment/human-values-and-ai-feedback-alignment/)

## Builds on (most cited first, 21 of 21)

- [Training language models to follow instructions with human feedback](https://mlsanity.com/paper/2203.02155/) (2022)
- [LoRA: Low-Rank Adaptation of Large Language Models](https://mlsanity.com/paper/2106.09685/) (2021)
- [Self-Instruct: Aligning Language Models with Self-Generated Instructions](https://mlsanity.com/paper/2212.10560/) (2022)
- [MPNet: Masked and Permuted Pre-training for Language Understanding](https://mlsanity.com/paper/2004.09297/) (2020)
- [BERTopic: Neural topic modeling with a class-based TF-IDF procedure](https://mlsanity.com/paper/2203.05794/) (2022)
- [Whose Opinions Do Language Models Reflect?](https://mlsanity.com/paper/2303.17548/) (2023)
- [Out of One, Many: Using Language Models to Simulate Human Samples](https://mlsanity.com/paper/2209.06899/) (2022)
- [Instruction Tuning for Large Language Models: A Survey](https://mlsanity.com/paper/2308.10792/) (2023)
- [Self-Play Fine-Tuning Converts Weak Language Models to Strong Language Models](https://mlsanity.com/paper/2401.01335/) (2024)
- [Towards Measuring the Representation of Subjective Global Opinions in Language Models](https://mlsanity.com/paper/2306.16388/) (2023)
- [The Butterfly Effect of Altering Prompts: How Small Changes and Jailbreaks Affect Large Language Model Performance](https://mlsanity.com/paper/2401.03729/) (2024)
- [Language Models Trained on Media Diets Can Predict Public Opinion](https://mlsanity.com/paper/2303.16779/) (2023)
- [How Susceptible are Large Language Models to Ideological Manipulation?](https://mlsanity.com/paper/2402.11725/) (2024)
- [Human-Instruction-Free LLM Self-Alignment with Limited Samples](https://mlsanity.com/paper/2401.06785/) (2024)
- [Whose Emotions and Moral Sentiments Do Language Models Reflect?](https://mlsanity.com/paper/2402.11114/) (2024)
- [Large Language Models as Subpopulation Representative Models: A Review](https://mlsanity.com/paper/2310.17888/) (2023)
- [Harmful Speech Detection by Language Models Exhibits Gender-Queer Dialect Bias](https://mlsanity.com/paper/2406.00020/) (2024)
- [Reading Between the Tweets: Deciphering Ideological Stances of Interconnected Mixed-Ideology Communities](https://mlsanity.com/paper/2402.01091/) (2024)
- [IsamasRed: A Public Dataset Tracking Reddit Discussions on Israel-Hamas Conflict](https://mlsanity.com/paper/2401.08202/) (2024)
- [Non-Binary Gender Expression in Online Interactions](https://mlsanity.com/paper/2303.04837/) (2023)
- [Polling Latent Opinions: A Method for Computational Sociolinguistics Using Transformer Language Models](https://mlsanity.com/paper/2204.07483/) (2022)

## Built on by (most cited first, 12 of 12)

- [Language Model Fine-Tuning on Scaled Survey Data for Predicting Distributions of Public Opinions](https://mlsanity.com/paper/2502.16761/) (2025)
- [Y Social: an LLM-powered Social Media Digital Twin](https://mlsanity.com/paper/2408.00818/) (2024)
- [STEER-BENCH: A Benchmark for Evaluating the Steerability of Large Language Models](https://mlsanity.com/paper/2505.20645/) (2025)
- [Leveraging Machine Learning to Identify Gendered Stereotypes and Body Image Concerns on Diet and Fitness Online Forums](https://mlsanity.com/paper/2407.03551/) (2024)
- [SubData: Bridging Heterogeneous Datasets to Enable Theory-Driven Evaluation of Political and Demographic Perspectives in LLMs](https://mlsanity.com/paper/2412.16783/) (2024)
- [MDIT: A Model-free Data Interpolation Method for Diverse Instruction Tuning](https://mlsanity.com/paper/2504.07288/) (2025)
- [Improving and Assessing the Fidelity of Large Language Models Alignment to Online Communities](https://mlsanity.com/paper/2408.09366/) (2024)
- [AlignSurvey: A Comprehensive Benchmark for Human Preferences Alignment in Social Surveys](https://mlsanity.com/paper/2511.07871/) (2025)
- [Synthia: Scalable Grounded Persona Generation from Social Media Data](https://mlsanity.com/paper/2507.14922/) (2025)
- [Graph-Based Alternatives to LLMs for Human Simulation](https://mlsanity.com/paper/2511.02135/) (2025)
- [Community-Aligned Behavior Under Uncertainty: Evidence of Epistemic Stance Transfer in LLMs](https://mlsanity.com/paper/2511.17572/) (2025)
- [Density-Guided Response Optimization: Community-Grounded Alignment via Implicit Acceptance Signals](https://mlsanity.com/paper/2603.03242/) (2026)

## Abstract (arXiv)

Social scientists use surveys to probe the opinions and beliefs of populations, but these methods are slow, costly, and prone to biases. Recent advances in large language models (LLMs) enable the creating of computational representations or "digital twins" of populations that generate human-like responses mimicking the population's language, styles, and attitudes. We introduce Community-Cross-Instruct, an unsupervised framework for aligning LLMs to online communities to elicit their beliefs. Given a corpus of a community's online discussions, Community-Cross-Instruct automatically generates instruction-output pairs by an advanced LLM to (1) finetune a foundational LLM to faithfully represent that community, and (2) evaluate the alignment of the finetuned model to the community. We demonstrate the method's utility in accurately representing political and diet communities on Reddit. Unlike prior methods requiring human-authored instructions, Community-Cross-Instruct generates instructions in a fully unsupervised manner, enhancing scalability and generalization across domains. This work enables cost-effective and automated surveying of diverse online communities.