mlsanityData to 7 Oct 2026

Social Learning: Towards Collaborative Learning with Large Language Models

arXiv 2312.11441 ↗First posted 18 Dec 2023cs.LG cs.CLcited by 15

Not on the map. Why?

Abstract (arXiv)

We introduce the framework of "social learning" in the context of large language models (LLMs), whereby models share knowledge with each other in a privacy-aware manner using natural language. We present and evaluate two approaches for knowledge transfer between LLMs. In the first scenario, we allow the model to generate abstract prompts aiming to teach the task. In our second approach, models transfer knowledge by generating synthetic examples. We evaluate these methods across diverse datasets and quantify memorization as a proxy for privacy loss. These techniques inspired by social learning yield promising results with low memorization of the original data. In particular, we show that performance using these methods is comparable to results with the use of original labels and prompts. Our work demonstrates the viability of social learning for LLMs, establishes baseline approaches and highlights several unexplored areas for future work.

Key facts. Cited by 15 papers in the map (data through 7 Oct 2026). Builds on 29 of its 43 references that are in the map. First posted on arXiv 18 Dec 2023.

Builds on

29 of 43 references.

References · 29

  1. 2020Language Models are Few-Shot Learnerscited by 32,199
  2. 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,302
  3. 2021Training Verifiers to Solve Math Word Problemscited by 9,743
  4. 2023Direct Preference Optimization: Your Language Model is Secretly a Reward Modelcited by 7,916
  5. 2016Communication-Efficient Learning of Deep Networks from Decentralized Datacited by 7,592
  6. 2021Finetuned Language Models Are Zero-Shot Learnerscited by 3,747
  7. 2019Advances and Open Problems in Federated Learningcited by 3,021
  8. 2023A Survey on Large Language Model based Autonomous Agentscited by 2,203
  9. 2019BoolQ: Exploring the Surprising Difficulty of Natural Yes/No Questionscited by 2,140
  10. 2023Improving Factuality and Reasoning in Language Models through Multiagent Debatecited by 1,739
  11. 2016Federated Learning: Strategies for Improving Communication Efficiencycited by 1,461
  12. 2022Rethinking the Role of Demonstrations: What Makes In-Context Learning Work?cited by 1,413
17 more
  1. 2021Scaling Language Models: Methods, Analysis & Insights from Training Gophercited by 1,199
  2. 2023PaLM 2 Technical Reportcited by 1,154
  3. 2020Adaptive Federated Optimizationcited by 1,036
  4. 2022Language Models as Zero-Shot Planners: Extracting Actionable Knowledge for Embodied Agentscited by 1,029
  5. 2023Encouraging Divergent Thinking in Large Language Models through Multi-Agent Debatecited by 995
  6. 2022Red Teaming Language Models with Language Modelscited by 983
  7. 2016The LAMBADA dataset: Word prediction requiring a broad discourse contextcited by 960
  8. 2018The Secret Sharer: Evaluating and Testing Unintended Memorization in Neural Networkscited by 847
  9. 2023Large Language Models as Optimizerscited by 830
  10. 2021Cross-Task Generalization via Natural Language Crowdsourcing Instructionscited by 696
  11. 2022Socratic Models: Composing Zero-Shot Multimodal Reasoning with Languagecited by 513
  12. 2022TALM: Tool Augmented Language Modelscited by 197
  13. 2022Instruction Induction: From Few Examples to Natural Language Task Descriptionscited by 152
  14. 2023Rephrase and Respond: Let Large Language Models Ask Better Questions for Themselvescited by 123
  15. 2023BOLAA: Benchmarking and Orchestrating LLM-augmented Autonomous Agentscited by 83
  16. 2023Synthetic Prompting: Generating Chain-of-Thought Demonstrations for Large Language Modelscited by 61
  17. 2022Conversational Information Seekingcited by 36

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 6; 2025: 6; 2026: 3.

15 papers.

Sort:

By area:

  1. 2024Cultural Evolution of Cooperation among LLM Agentscited by 30
  2. 2025Unmasking Conversational Bias in AI Multiagent Systemscited by 16
  3. 2025Don't lie to your friends: Learning what you know from collaborative self-playcited by 14
  4. 2024Can LLMs get help from other LLMs without revealing private information?cited by 9
  5. 2024On-Device Collaborative Language Modeling via a Mixture of Generalists and Specialistscited by 6
  6. 2026Improving Interactive In-Context Learning from Natural Language Feedbackcited by 5
  7. 2025Trace-of-Thought Prompting: Investigating Prompt-Based Knowledge Distillation Through Question Decompositioncited by 3
  8. 2024Interacting Large Language Model Agents. Interpretable Models and Social Learningcited by 2
  9. 2026Federation over Text: Insight Sharing for Multi-Agent Reasoningcited by 2
  10. 2024CollabEdit: Towards Non-destructive Collaborative Knowledge Editingcited by 1
  11. 2025Attention is All You Need Until You Need Retentioncited by 1
  12. 2024Distributed In-Context Learning under Non-IID Among Clientscited by 0
3 more
  1. 2025Towards Interpretable Renal Health Decline Forecasting via Multi-LMM Collaborative Reasoning Frameworkcited by 0
  2. 2025Expertise-aware Multi-LLM Recruitment and Collaboration for Medical Decision-Makingcited by 0
  3. 2026From Solo to Social Learning: Characterizing Recursive Social Improvement in LLMscited by 0