mlsanityData to 14 Oct 2026

SOTOPIA-π: Interactive Learning of Socially Intelligent Language Agents

arXiv 2403.08715 ↗First posted 13 Mar 2024cs.CLcited by 93

On the map: LLM agents and tool use → General LLM agent frameworks and benchmarks

Abstract (arXiv)

Humans learn social skills through both imitation and social interaction. This social learning process is largely understudied by existing research on building language agents. Motivated by this gap, we propose an interactive learning method, SOTOPIA-π, improving the social intelligence of language agents. This method leverages behavior cloning and self-reinforcement training on filtered social interaction data according to large language model (LLM) ratings. We show that our training method allows a 7B LLM to reach the social goal completion ability of an expert model (GPT-4-based agent), while improving the safety of language agents and maintaining general QA ability on the MMLU benchmark. We also find that this training paradigm uncovers some difficulties in LLM-based evaluation of social intelligence: LLM-based evaluators overestimate the abilities of the language agents trained specifically for social interaction.

Key facts. Cited by 93 papers in the map (data through 14 Oct 2026). Placed in General LLM agent frameworks and benchmarks, one of 16 directions in LLM agents and tool use. Builds on 42 of its 61 references that are in the map. First posted on arXiv 13 Mar 2024.

Builds on

42 of 61 references.

References · 42

  1. 2022Training language models to follow instructions with human feedbackcited by 15,971
  2. 2017Proximal Policy Optimization Algorithmscited by 15,215
  3. 2021LoRA: Low-Rank Adaptation of Large Language Modelscited by 14,380
  4. 2023Direct Preference Optimization: Your Language Model is Secretly a Reward Modelcited by 8,065
  5. 2020Measuring Massive Multitask Language Understandingcited by 7,963
  6. 2021Prefix-Tuning: Optimizing Continuous Prompts for Generationcited by 3,915
  7. 2017Deep reinforcement learning from human preferencescited by 3,912
  8. 2021The Power of Scale for Parameter-Efficient Prompt Tuningcited by 3,795
  9. 2023QLoRA: Efficient Finetuning of Quantized LLMscited by 3,418
  10. 2022Constitutional AI: Harmlessness from AI Feedbackcited by 3,050
  11. 2019Fine-Tuning Language Models from Human Preferencescited by 2,150
  12. 2023Large Language Models are not Fair Evaluatorscited by 1,085
30 more
  1. 2021Ethical and social risks of harm from Language Modelscited by 966
  2. 2023A General Theoretical Paradigm to Understand Learning from Human Preferencescited by 912
  3. 2023RAFT: Reward rAnked FineTuning for Generative Foundation Model Alignmentcited by 686
  4. 2023An Empirical Study of Catastrophic Forgetting in Large Language Models During Continual Fine-tuningcited by 639
  5. 2023Bias and Fairness in Large Language Models: A Surveycited by 639
  6. 2023DecodingTrust: A Comprehensive Assessment of Trustworthiness in GPT Modelscited by 518
  7. 2023Reinforced Self-Training (ReST) for Language Modelingcited by 500
  8. 2018Behavioral Cloning from Observationcited by 492
  9. 2023Trustworthy LLMs: a Survey and Guideline for Evaluating Large Language Models' Alignmentcited by 401
  10. 2023Large Language Models Fail on Trivial Alterations to Theory-of-Mind Taskscited by 292
  11. 2020Social Chemistry 101: Learning to Reason about Social and Moral Normscited by 255
  12. 2023Can LLMs Keep a Secret? Testing Privacy Implications of Language Models via Contextual Integrity Theorycited by 230
  13. 2023An Overview of Catastrophic AI Riskscited by 222
  14. 2022Neural Theory-of-Mind? On the Limits of Social Intelligence in Large LMscited by 220
  15. 2021Open-Ended Learning Leads to Generally Capable Agentscited by 182
  16. 2023Understanding Social Reasoning in Language Models with Language Modelscited by 172
  17. 2023Clever Hans or Neural Theory of Mind? Stress Testing Social Reasoning in Large Language Modelscited by 163
  18. 2023Cultural Bias and Cultural Alignment of Large Language Modelscited by 149
  19. 2018Self-Imitation Learningcited by 138
  20. 2019Gmail Smart Compose: Real-Time Assisted Writingcited by 98
  21. 2023Training Socially Aligned Language Models on Simulated Social Interactionscited by 78
  22. 2021Towards Facilitating Empathic Conversations in Online Mental Health Support: A Reinforcement Learning Approachcited by 75
  23. 2022X-Risk Analysis for AI Researchcited by 60
  24. 2023NormBank: A Knowledge Bank of Situational Social Normscited by 60
  25. 2023AvalonBench: Evaluating LLMs Playing the Game of Avaloncited by 47
  26. 2022Why So Toxic? Measuring and Triggering Toxic Behavior in Open-Domain Chatbotscited by 33
  27. 2021Few-shot Language Coordination by Modeling Theory of Mindcited by 32
  28. 2023An In-depth Look at Gemini's Language Abilitiescited by 28
  29. 2023Computational Language Acquisition with Theory of Mindcited by 15
  30. 2023Self-Supervised Behavior Cloned Transformers are Path Crawlers for Text Gamescited by 1

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2024: 18; 2025: 34; 2026: 41.

93 papers.

Sort:

By area:

  1. 2024From Persona to Personalization: A Survey on Role-Playing Language Agentscited by 212
  2. 2024A Survey on Large Language Model-Based Game Agentscited by 112
  3. 2024A Survey on Self-Evolution of Large Language Modelscited by 91
  4. 2024From Individual to Society: A Survey on Social Simulation Driven by Large Language Model-based Agentscited by 82
  5. 2024Towards Implicit Bias Detection and Mitigation in Multi-Agent LLM Interactionscited by 51
  6. 2025Agentic Reinforcement Learning with Implicit Step Rewardscited by 47
  7. 2025From Language to Action: A Review of Large Language Models as Autonomous Agents and Tool Userscited by 32
  8. 2024AI-LieDar: Examine the Trade-off Between Utility and Truthfulness in LLM Agentscited by 31
  9. 2025ATLaS: Agent Tuning via Learning Critical Stepscited by 31
  10. 2025SDPO: Segment-Level Direct Preference Optimization for Social Agentscited by 30
  11. 2025Amulet: ReAlignment During Test Time for Personalized Preference Adaptation of LLMscited by 29
  12. 2025OPeRA: A Dataset of Observation, Persona, Rationale, and Action for Evaluating LLMs on Human Online Shopping Behavior Simulationcited by 28
38 more
  1. 2025MOSAIC: Modeling Social AI for Content Dissemination and Regulation in Multi-Agent Simulationscited by 27
  2. 2024Examining Identity Drift in Conversations of LLM Agentscited by 25
  3. 2025Sotopia-RL: Reward Design for Social Intelligencecited by 24
  4. 2026CooperBench: Why Coding Agents Cannot be Your Teammates Yetcited by 23
  5. 2024PATIENT-Psi: Using Large Language Models to Simulate Patients for Training Mental Health Professionalscited by 21
  6. 2024MMRole: A Comprehensive Framework for Developing and Evaluating Multimodal Role-Playing Agentscited by 21
  7. 2025EmoAgent: Assessing and Safeguarding Human-AI Interaction for Mental Health Safetycited by 21
  8. 2025Sentient Agent as a Judge: Evaluating Higher-Order Social Cognition in Large Language Modelscited by 21
  9. 2025SIV-Bench: A Video Benchmark for Social Interaction Understanding and Reasoningcited by 18
  10. 2025One Size doesn't Fit All: A Personalized Conversational Tutoring Agent for Mathematics Instructioncited by 17
  11. 2024A Systematic Assessment of OpenAI o1-Preview for Higher Order Thinking in Educationcited by 16
  12. 2025SOTOPIA-Omega: Dynamic Strategy Injection Learning and Social Instruction Following Evaluation for Social Agentscited by 16
  13. 2025Cultural Learning-Based Culture Adaptation of Language Modelscited by 16
  14. 2025Social World Modelscited by 16
  15. 2024Efficacy of Language Model Self-Play in Non-Zero-Sum Gamescited by 15
  16. 2025Adaptive Social Learning via Mode Policy Optimization for Language Agentscited by 15
  17. 2025Don't lie to your friends: Learning what you know from collaborative self-playcited by 14
  18. 2026PivotRL: High Accuracy Agentic Post-Training at Low Compute Costcited by 14
  19. 2025EgoNormia: Benchmarking Physical Social Norm Understandingcited by 13
  20. 2026OdysSim: Building Foundation Models for Human Behavior Simulationcited by 13
  21. 2024The Imperative of Conversation Analysis in the Era of LLMs: A Survey of Tasks, Techniques, and Trendscited by 11
  22. 2025Multi-agent KTO: Reinforcing Strategic Interactions of Large Language Model in Language Gamecited by 10
  23. 2025SOTOPIA-S4: a user-friendly system for flexible, customizable, and large-scale social simulationcited by 10
  24. 2024Revealing and Mitigating the Challenge of Detecting Character Knowledge Errors in LLM Role-Playingcited by 9
  25. 2025LIFELONG SOTOPIA: Evaluating Social Intelligence of Language Agents Over Lifelong Social Interactionscited by 8
  26. 2025Infusing Theory of Mind into Socially Intelligent LLM Agentscited by 8
  27. 2026M3-BENCH: Process-Aware Evaluation of LLM Agents' Social Behaviors in Mixed-Motive Gamescited by 8
  28. 2026One Model, All Roles: Multi-Turn, Multi-Agent Self-Play Reinforcement Learning for Conversational Social Intelligencecited by 7
  29. 2024DEMO: Reframing Dialogue Interaction with Fine-grained Element Modelingcited by 6
  30. 2025MIRAGE: Exploring How Large Language Models Perform in Complex Social Interactive Environmentscited by 6
  31. 2025HARBOR: Exploring Persona Dynamics in Multi-Agent Competitioncited by 6
  32. 2025If an LLM Were a Character, Would It Know Its Own Story? Evaluating Lifelong Learning in LLMscited by 5
  33. 2025Multi-level Value Alignment in Agentic AI Systems: Survey and Perspectivescited by 5
  34. 2024Fostering Natural Conversation in Large Language Models with NICO: a Natural Interactive COnversation datasetcited by 4
  35. 2026Implicit Turn-Wise Policy Optimization for Proactive User-LLM Interactioncited by 4
  36. 2025Looking Beyond the Obvious: A Survey on Abstract Concept Recognition for Video Understandingcited by 3
  37. 2025Evalet: Evaluating Large Language Models through Functional Fragmentationcited by 3
  38. 2025The Stackelberg Speaker: Optimizing Persuasive Communication in Social Deduction Gamescited by 3

50 most cited of 93 All 93 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2023SOTOPIA: Interactive Evaluation for Social Intelligence in Language Agents
  2. 2024AgentSense: Benchmarking Social Intelligence of Language Agents through Interactive Scenarios
  3. 2024Social Learning through Interactions with Other Agents: A Survey
  4. 2024Towards Objectively Benchmarking Social Intelligence for Language Agents at Action Level
  5. 2026From Solo to Social Learning: Characterizing Recursive Social Improvement in LLMs
  6. 2026Playing social deduction games with reinforcement fine-tuned large language models
  7. 2025Talking with Oompa Loompas: A novel framework for evaluating linguistic acquisition of LLM agents
  8. 2026Social World Model for Lifelong Social Intelligence