# Social Learning: Towards Collaborative Learning with Large Language Models

arXiv: https://arxiv.org/abs/2312.11441 · HTML: https://mlsanity.com/paper/2312.11441/

## Key facts

- Cited by 15 papers in the map (data through 14 Oct 2026).
- Builds on 29 of its 43 references that are in the map.
- First posted on arXiv 18 Dec 2023.

## Builds on (most cited first, 29 of 29)

- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [Chain-of-Thought Prompting Elicits Reasoning in Large Language Models](https://mlsanity.com/paper/2201.11903/) (2022)
- [Training Verifiers to Solve Math Word Problems](https://mlsanity.com/paper/2110.14168/) (2021)
- [Direct Preference Optimization: Your Language Model is Secretly a Reward Model](https://mlsanity.com/paper/2305.18290/) (2023)
- [Communication-Efficient Learning of Deep Networks from Decentralized Data](https://mlsanity.com/paper/1602.05629/) (2016)
- [Finetuned Language Models Are Zero-Shot Learners](https://mlsanity.com/paper/2109.01652/) (2021)
- [Advances and Open Problems in Federated Learning](https://mlsanity.com/paper/1912.04977/) (2019)
- [A Survey on Large Language Model based Autonomous Agents](https://mlsanity.com/paper/2308.11432/) (2023)
- [BoolQ: Exploring the Surprising Difficulty of Natural Yes/No Questions](https://mlsanity.com/paper/1905.10044/) (2019)
- [Improving Factuality and Reasoning in Language Models through Multiagent Debate](https://mlsanity.com/paper/2305.14325/) (2023)
- [Federated Learning: Strategies for Improving Communication Efficiency](https://mlsanity.com/paper/1610.05492/) (2016)
- [Rethinking the Role of Demonstrations: What Makes In-Context Learning Work?](https://mlsanity.com/paper/2202.12837/) (2022)
- [Scaling Language Models: Methods, Analysis & Insights from Training Gopher](https://mlsanity.com/paper/2112.11446/) (2021)
- [PaLM 2 Technical Report](https://mlsanity.com/paper/2305.10403/) (2023)
- [Adaptive Federated Optimization](https://mlsanity.com/paper/2003.00295/) (2020)
- [Language Models as Zero-Shot Planners: Extracting Actionable Knowledge for Embodied Agents](https://mlsanity.com/paper/2201.07207/) (2022)
- [Red Teaming Language Models with Language Models](https://mlsanity.com/paper/2202.03286/) (2022)
- [Encouraging Divergent Thinking in Large Language Models through Multi-Agent Debate](https://mlsanity.com/paper/2305.19118/) (2023)
- [The LAMBADA dataset: Word prediction requiring a broad discourse context](https://mlsanity.com/paper/1606.06031/) (2016)
- [The Secret Sharer: Evaluating and Testing Unintended Memorization in Neural Networks](https://mlsanity.com/paper/1802.08232/) (2018)
- [Large Language Models as Optimizers](https://mlsanity.com/paper/2309.03409/) (2023)
- [Cross-Task Generalization via Natural Language Crowdsourcing Instructions](https://mlsanity.com/paper/2104.08773/) (2021)
- [Socratic Models: Composing Zero-Shot Multimodal Reasoning with Language](https://mlsanity.com/paper/2204.00598/) (2022)
- [TALM: Tool Augmented Language Models](https://mlsanity.com/paper/2205.12255/) (2022)
- [Instruction Induction: From Few Examples to Natural Language Task Descriptions](https://mlsanity.com/paper/2205.10782/) (2022)
- [Rephrase and Respond: Let Large Language Models Ask Better Questions for Themselves](https://mlsanity.com/paper/2311.04205/) (2023)
- [BOLAA: Benchmarking and Orchestrating LLM-augmented Autonomous Agents](https://mlsanity.com/paper/2308.05960/) (2023)
- [Synthetic Prompting: Generating Chain-of-Thought Demonstrations for Large Language Models](https://mlsanity.com/paper/2302.00618/) (2023)
- [Conversational Information Seeking](https://mlsanity.com/paper/2201.08808/) (2022)

## Built on by (most cited first, 15 of 15)

- [Cultural Evolution of Cooperation among LLM Agents](https://mlsanity.com/paper/2412.10270/) (2024)
- [Unmasking Conversational Bias in AI Multiagent Systems](https://mlsanity.com/paper/2501.14844/) (2025)
- [Don't lie to your friends: Learning what you know from collaborative self-play](https://mlsanity.com/paper/2503.14481/) (2025)
- [Can LLMs get help from other LLMs without revealing private information?](https://mlsanity.com/paper/2404.01041/) (2024)
- [On-Device Collaborative Language Modeling via a Mixture of Generalists and Specialists](https://mlsanity.com/paper/2409.13931/) (2024)
- [Improving Interactive In-Context Learning from Natural Language Feedback](https://mlsanity.com/paper/2602.16066/) (2026)
- [Trace-of-Thought Prompting: Investigating Prompt-Based Knowledge Distillation Through Question Decomposition](https://mlsanity.com/paper/2504.20946/) (2025)
- [Federation over Text: Insight Sharing for Multi-Agent Reasoning](https://mlsanity.com/paper/2604.16778/) (2026)
- [Interacting Large Language Model Agents. Interpretable Models and Social Learning](https://mlsanity.com/paper/2411.01271/) (2024)
- [CollabEdit: Towards Non-destructive Collaborative Knowledge Editing](https://mlsanity.com/paper/2410.09508/) (2024)
- [Attention is All You Need Until You Need Retention](https://mlsanity.com/paper/2501.09166/) (2025)
- [Distributed In-Context Learning under Non-IID Among Clients](https://mlsanity.com/paper/2408.00144/) (2024)
- [Towards Interpretable Renal Health Decline Forecasting via Multi-LMM Collaborative Reasoning Framework](https://mlsanity.com/paper/2507.22464/) (2025)
- [Expertise-aware Multi-LLM Recruitment and Collaboration for Medical Decision-Making](https://mlsanity.com/paper/2508.13754/) (2025)
- [From Solo to Social Learning: Characterizing Recursive Social Improvement in LLMs](https://mlsanity.com/paper/2609.38516/) (2026)

## Abstract (arXiv)

We introduce the framework of "social learning" in the context of large language models (LLMs), whereby models share knowledge with each other in a privacy-aware manner using natural language. We present and evaluate two approaches for knowledge transfer between LLMs. In the first scenario, we allow the model to generate abstract prompts aiming to teach the task. In our second approach, models transfer knowledge by generating synthetic examples. We evaluate these methods across diverse datasets and quantify memorization as a proxy for privacy loss. These techniques inspired by social learning yield promising results with low memorization of the original data. In particular, we show that performance using these methods is comparable to results with the use of original labels and prompts. Our work demonstrates the viability of social learning for LLMs, establishes baseline approaches and highlights several unexplored areas for future work.