mlsanityData to 7 Oct 2026

MineLand: Simulating Large-Scale Multi-Agent Interactions with Limited Multimodal Senses and Physical Needs

arXiv 2403.19267 ↗First posted 28 Mar 2024cs.CL cs.AIcited by 16

On the map: LLM agents and tool use → Reinforcement learning for LLM agents

Abstract (arXiv)

While Vision-Language Models (VLMs) hold promise for tasks requiring extensive collaboration, traditional multi-agent simulators have facilitated rich explorations of an interactive artificial society that reflects collective behavior. However, these existing simulators face significant limitations. Firstly, they struggle with handling large numbers of agents due to high resource demands. Secondly, they often assume agents possess perfect information and limitless capabilities, hindering the ecological validity of simulated social interactions. To bridge this gap, we propose a multi-agent Minecraft simulator, MineLand, that bridges this gap by introducing three key features: large-scale scalability, limited multimodal senses, and physical needs. Our simulator supports 64 or more agents. Agents have limited visual, auditory, and environmental awareness, forcing them to actively communicate and collaborate to fulfill physical needs like food and resources. Additionally, we further introduce an AI agent framework, Alex, inspired by multitasking theory, enabling agents to handle intricate coordination and scheduling. Our experiments demonstrate that the simulator, the corresponding benchmark, and the AI agent framework contribute to more ecological and nuanced collective behavior.The source code of MineLand and Alex is openly available at https://github.com/cocacola-lab/MineLand.

Key facts. Cited by 16 papers in the map (data through 7 Oct 2026). Placed in Reinforcement learning for LLM agents, one of 16 directions in LLM agents and tool use. Builds on 31 of its 61 references that are in the map. First posted on arXiv 28 Mar 2024.

Builds on

31 of 61 references.

Closest ancestors · 5

Same direction, newest first

  1. 2024Hierarchical Auto-Organizing System for Open-Ended Multi-Agent Navigation13 Mar
  2. 2023JARVIS-1: Open-World Multi-task Agents with Memory-Augmented Multimodal Language Models10 Nov
  3. 2023Ghost in the Minecraft: Generally Capable Agents for Open-World Environments via Large Language Models with Text-based Knowledge and Memory25 May
  4. 2023Voyager: An Open-Ended Embodied Agent with Large Language Models25 May
  5. 2022MineDojo: Building Open-Ended Embodied Agents with Internet-Scale Knowledge17 Jun

Other references · 26

  1. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
  2. 2023Generative Agents: Interactive Simulacra of Human Behaviorcited by 3,500
  3. 2017AI2-THOR: An Interactive 3D Environment for Visual AIcited by 850
  4. 2020SAPIEN: A SimulAted Part-based Interactive ENvironmentcited by 584
  5. 2021Habitat 2.0: Training Home Assistants to Rearrange their Habitatcited by 501
  6. 2018VirtualHome: Simulating Household Activities via Programscited by 411
  7. 2023Describe, Explain, Plan and Select: Interactive Planning with Large Language Models Enables Open-World Multi-Task Agentscited by 396
  8. 2022Using cognitive psychology to understand GPT-3cited by 389
  9. 2022Social Simulacra: Creating Populated Prototypes for Social Computing Systemscited by 300
  10. 2023Building Cooperative Embodied Agents Modularly with Large Language Modelscited by 231
  11. 2023Habitat 3.0: A Co-Habitat for Humans, Avatars and Robotscited by 220
  12. 2019MineRL: A Large-Scale Dataset of Minecraft Demonstrationscited by 210
14 more
  1. 2021iGibson 2.0: Object-Centric Simulation for Robot Learning of Everyday Household Taskscited by 191
  2. 2020iGibson 1.0: a Simulation Environment for Interactive Tasks in Large Realistic Scenescited by 151
  3. 2020Watch-And-Help: A Challenge for Social Perception and Human-AI Collaborationcited by 131
  4. 2022Evaluating and Inducing Personality in Pre-trained Language Modelscited by 122
  5. 2023Mindstorms in Natural Language-Based Societies of Mindcited by 105
  6. 2021Scalable Evaluation of Multi-Agent Reinforcement Learning with Melting Potcited by 97
  7. 2023MindAgent: Emergent Gaming Interactioncited by 87
  8. 2023KoLA: Carefully Benchmarking World Knowledge of Large Language Modelscited by 85
  9. 2023MP5: A Multi-modal Open-ended Embodied System in Minecraft via Active Perceptioncited by 67
  10. 2023Epidemic Modeling with Generative Agentscited by 49
  11. 2023Open-World Multi-Task Control Through Goal-Aware Representation Learning and Adaptive Horizon Predictioncited by 39
  12. 2019The Multi-Agent Reinforcement Learning in Malm"O (MARL"O) Competitioncited by 14
  13. 2023Is There Any Social Principle for LLM-Based Agents?cited by 4
  14. 2023Exploring Trust and Risk during Online Bartering Interactionscited by 1

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 6; 2025: 5; 2026: 5.

16 papers.

Sort:

By area:

  1. 2024OASIS: Open Agent Social Interaction Simulations with One Million Agentscited by 127
  2. 2024From Individual to Society: A Survey on Social Simulation Driven by Large Language Model-based Agentscited by 76
  3. 2026A Survey of Agentic Reasoning for Large Language Models: Towards Recursively Self-Improving and Collective Agentscited by 47
  4. 2024TeamCraft: A Benchmark for Multi-Modal Multi-Agent Systems in Minecraftcited by 20
  5. 2024LMAgent: A Large-scale Multimodal Agents Society for Multi-user Simulationcited by 14
  6. 2025Virtual Community: An Open World for Humans, Robots, and Societycited by 13
  7. 2025SimWorld: An Open-ended Realistic Simulator for Autonomous Agents in Physical and Social Worldscited by 10
  8. 2026The Five Ws of Multi-Agent Communication: Who Talks to Whom, When, What, and Why -- A Survey from MARL to Emergent Language and LLMscited by 10
  9. 2024JaxLife: An Open-Ended Agentic Simulatorcited by 6
  10. 2025Ella: Embodied Social Agents with Lifelong Memorycited by 4
  11. 2025PillagerBench: Benchmarking LLM-Based Agents in Competitive Minecraft Team Environmentscited by 4
  12. 2026MECoBench: A Systematic Study of Multimodal Agent Collaboration in Embodied Environmentscited by 2
4 more
  1. 2024APT: Architectural Planning and Text-to-Blueprint Construction Using Large Language Models for Open-World Agentscited by 1
  2. 2026Multi-agent Framework for Time-Sensitive Complementary Collaboration in Minecraftcited by 1
  3. 2025IndoorWorld: Integrating Physical Task Solving and Social Simulation in A Heterogeneous Multi-Agent Environmentcited by 0
  4. 2026Lies We Can See: Joint Verbal and Non-Verbal Deception by VLM Agents in Embodied Social Interactionscited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2026MineExplorer: Evaluating Open-World Exploration of MLLM Agents in Minecraft
  2. 2024Project Sid: Many-agent simulations toward AI civilization
  3. 2024STEVE Series: Step-by-Step Construction of Agent Systems in Minecraft
  4. 2025Static Sandboxes Are Inadequate: Modeling Societal Complexity Requires Open-Ended Co-Evolution in LLM-Based Multi-Agent Simulations
  5. 2025OpenHA: A Series of Open-Source Hierarchical Agentic Models in Minecraft
  6. 2025CrafterDojo: A Suite of Foundation Models for Building Open-Ended Embodied Agents in Crafter
  7. 2024Odyssey: Empowering Minecraft Agents with Open-World Skills
  8. 2024Craftium: Bridging Flexibility and Efficiency for Rich 3D Single- and Multi-Agent Environments