mlsanityData to 14 Oct 2026

SMAC-Talk: A Natural Language Extension of the StarCraft Multi-Agent Challenge for Large Language Models

arXiv 2606.04202 ↗First posted 2 Jun 2026cs.AIcited by 0

On the map: LLM agents and tool use → General LLM agent frameworks and benchmarks

Abstract (arXiv)

As LLMs become more widely deployed, they are increasingly expected to work alongside other AI agents rather than operating in isolation. Effective coordination in these settings requires agents to communicate, share information and make decisions under uncertainty. We introduce SMAC-Talk, a natural language extension of the StarCraft Multi-Agent Challenge for evaluating LLM-based agents in cooperative multi-agent environments. The environment has several key features such as decentralized control, partial observability and long-horizon decision making. SMAC-Talk includes a natural language communication channel which is used to probe agent coordination and trust. We use this communication channel to construct different evaluation scenarios, including settings with an embedded deceptive communicator that tries to disrupt and deceive allies through communication alone. We provide three agents for benchmarking using 4 models from the Qwen3.5 family and study how reasoning structure, memory and model scale affect coordination between agents. We release SMAC-Talk as an open benchmark to support the research community in developing and evaluating LLM agents in cooperative multi-agent settings.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in General LLM agent frameworks and benchmarks, one of 16 directions in LLM agents and tool use. Builds on 21 of its 25 references that are in the map. First posted on arXiv 2 Jun 2026.

Builds on

21 of 25 references.

Closest ancestors · 4

Same direction, newest first

  1. 2025Multi-Agent Collaboration Mechanisms: A Survey of LLMs10 Jan
  2. 2023AgentBench: Evaluating LLMs as Agents7 Aug
  3. 2023MetaGPT: Meta Programming for A Multi-Agent Collaborative Framework1 Aug
  4. 2023CAMEL: Communicative Agents for "Mind" Exploration of Large Language Model Society31 Mar

Other references · 17

  1. 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,459
  2. 2022ReAct: Synergizing Reasoning and Acting in Language Modelscited by 8,695
  3. 2023Efficient Memory Management for Large Language Model Serving with PagedAttentioncited by 5,944
  4. 2023Voyager: An Open-Ended Embodied Agent with Large Language Modelscited by 2,191
  5. 2020ALFWorld: Aligning Text and Embodied Environments for Interactive Learningcited by 1,205
  6. 2019The StarCraft Multi-Agent Challengecited by 597
  7. 2016Learning to Communicate with Deep Multi-Agent Reinforcement Learningcited by 575
  8. 2016Learning Multiagent Communication with Backpropagationcited by 366
  9. 2023Playing repeated games with Large Language Modelscited by 217
  10. 2022SMACv2: An Improved Benchmark for Cooperative Multi-Agent Reinforcement Learningcited by 122
  11. 2023Large Language Models Play StarCraft II: Benchmarks and A Chain of Summarization Approachcited by 79
  12. 2023ChessGPT: Bridging Policy Learning and Language Modelingcited by 75
5 more
  1. 2024How Far Are We on the Decision-Making of LLMs? Evaluating LLMs' Gaming Ability in Multi-Agent Environmentscited by 67
  2. 2025The Traitors: Deception and Trust in Multi-Agent Language Model Simulationscited by 34
  3. 2025LLM-Hanabi: Evaluating Multi-Agent Gameplays with Theory-of-Mind and Rationale Inference in Imperfect Information Collaboration Gamecited by 8
  4. 2025LH-Deception: Simulating and Understanding LLM Deceptive Behaviors in Long-Horizon Interactionscited by 7
  5. 2025HLSMAC: A New StarCraft Multi-Agent Challenge for High-Level Strategic Decision-Makingcited by 2

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2025Beyond Self-Talk: A Communication-Centric Survey of LLM-Based Multi-Agent Systems
  2. 2024Large Language Model based Multi-Agents: A Survey of Progress and Challenges
  3. 2025Communication and Verification in LLM Agents towards Collaboration under Information Asymmetry
  4. 2024A Survey on Large Language Model-Based Game Agents
  5. 2025Grounding Natural Language for Multi-agent Decision-Making with Multi-agentic LLMs
  6. 2023LLM-Coordination: Evaluating and Analyzing Multi-agent Coordination Abilities in Large Language Models
  7. 2025Systematic Failures in Collective Reasoning under Distributed Information in Multi-Agent LLMs
  8. 2026Competition and Cooperation of LLM Agents in Games