mlsanityData to 7 Oct 2026

SocialGrid: A Benchmark for Planning and Social Reasoning in Embodied Multi-Agent Systems

arXiv 2604.16022 ↗First posted 17 Apr 2026cs.AI cs.LG cs.MAcited by 0

On the map: LLM reasoning → Reasoning for agents, planning and code

also in 1 more
Abstract (arXiv)

As Large Language Models (LLMs) transition from text processors to autonomous agents, evaluating their social reasoning in embodied multi-agent settings becomes critical. We introduce SocialGrid, an embodied multi-agent environment inspired by Among Us that evaluates LLM agents on planning, task execution, and social reasoning. Our evaluations reveal that even the strongest open model (GPT-OSS-120B) achieves below 60% accuracy in task completion and planning, with agents getting stuck in repetitive behaviors or failing to navigate basic obstacles. Since poor navigation confounds evaluation of social intelligence, SocialGrid offers an optional Planning Oracle to isolate social reasoning from planning deficits. While planning assistance improves task completion, social reasoning remains a bottleneck: agents fail to detect deception at near-random chance regardless of scale, relying on shallow heuristics rather than accumulating behavioral evidence. SocialGrid provides automatic failure analysis and fine-grained metrics, enabling developers to diagnose and improve their agents. We also establish a competitive leaderboard using Elo ratings from adversarial league play.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Placed in Reasoning for agents, planning and code, one of 17 directions in LLM reasoning. Builds on 21 of its 52 references that are in the map. First posted on arXiv 17 Apr 2026.

Builds on

21 of 52 references.

References · 21

  1. 2017Proximal Policy Optimization Algorithmscited by 14,441
  2. 2024The Llama 3 Herd of Modelscited by 14,192
  3. 2025Qwen3 Technical Reportcited by 8,195
  4. 2025gpt-oss-120b & gpt-oss-20b Model Cardcited by 1,258
  5. 2024Phi-4 Technical Reportcited by 675
  6. 2023Exploring Large Language Models for Communication Games: An Empirical Study on Werewolfcited by 238
  7. 2025Survey on Evaluation of LLM-based Agentscited by 190
  8. 2023EgoPlan-Bench: Benchmarking Multimodal Large Language Models for Human-Level Planningcited by 66
  9. 2023Hoodwinked: Deception and Cooperation in a Text-Based Game for Language Modelscited by 46
  10. 2023AvalonBench: Evaluating LLMs Playing the Game of Avaloncited by 45
  11. 2025SpatialBench: Benchmarking Multimodal Large Language Models for Spatial Cognitioncited by 35
  12. 2025Mind the Gap: Benchmarking Spatial Reasoning in Vision-Language Modelscited by 29
9 more
  1. 2024AMONGAGENTS: Evaluating Large Language Models in the Interactive Text-Based Social Deduction Gamecited by 26
  2. 2025Multi-Agent-as-Judge: Aligning LLM-Agent-Based Automated Evaluation with Multi-Dimensional Human Evaluationcited by 26
  3. 2025Among Us: A Sandbox for Measuring and Detecting Agentic Deceptioncited by 24
  4. 2025TAMAS: Benchmarking Adversarial Risks in Multi-Agent LLM Systemscited by 24
  5. 2025SocialMaze: A Benchmark for Evaluating and Enhancing Social Reasoning in Large Language Models in Complex Social Environmentscited by 9
  6. 2025CoopReflect: Towards Natural Language Communication for Cooperative Autonomous Driving via Multi-Agent Learningcited by 7
  7. 2025GridRoute: A Benchmark for LLM-Based Route Planning with Cardinal Movement in Grid Environmentscited by 4
  8. 2025Configurable multi-agent framework for scalable and realistic testing of llm-based agentscited by 2
  9. 2025ProbLog4Fairness: A Neurosymbolic Approach to Modeling and Mitigating Biascited by 1

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2024AgentSense: Benchmarking Social Intelligence of Language Agents through Interactive Scenarios
  2. 2024Towards Objectively Benchmarking Social Intelligence for Language Agents at Action Level
  3. 2025SPIN-Bench: How Well Do LLMs Plan Strategically and Reason Socially?
  4. 2026Social Gym and SPaRTan: Benchmarking and Improving LLM Social Reasoning via Multi-Agent Game Tournaments
  5. 2023SOTOPIA: Interactive Evaluation for Social Intelligence in Language Agents
  6. 2024PARTNR: A Benchmark for Planning and Reasoning in Embodied Multi-agent Tasks
  7. 2026MINDGAMES: A Live Arena for Evaluating Social and Strategic Reasoning in Multi-Agent LLMs
  8. 2026Communicate-Predict-Act: Evaluating Social Intelligence of Agents