OpenHospital: A Thing-in-itself Arena for Evolving and Benchmarking LLM-based Collective Intelligence
On the map: LLM agents and tool use → Medical and scientific domain agents
Abstract (arXiv)
Large Language Model (LLM)-based Collective Intelligence (CI) presents a promising approach to overcoming the data wall and continuously boosting the capabilities of LLM agents. However, there is currently no dedicated arena for evolving and benchmarking LLM-based CI. To address this gap, we introduce OpenHospital, an interactive arena where physician agents can evolve CI through interactions with patient agents. This arena employs a data-in-agent-self paradigm that rapidly enhances agent capabilities and provides robust evaluation metrics for benchmarking both medical proficiency and system efficiency. Experiments demonstrate the effectiveness of OpenHospital in both fostering and quantifying CI.
Key facts. Cited by 1 paper in the map (data through 14 Oct 2026). Placed in Medical and scientific domain agents, one of 16 directions in LLM agents and tool use. Builds on 16 of its 32 references that are in the map. First posted on arXiv 16 Mar 2026.
Builds on
16 of 32 references.
Closest ancestors · 3
Same direction, newest first
- 2025Self-Evolving Multi-Agent Simulations for Realistic Clinical Interactions28 Mar
- 2024AgentClinic: a multimodal agent benchmark to evaluate AI in simulated clinical environments13 May
- 2024Agent Hospital: A Simulacrum of Hospital with Evolvable Medical Agents5 May
Other references · 13
- 2023Generative Agents: Interactive Simulacra of Human Behaviorcited by 3,568
- 2023SWE-bench: Can Language Models Resolve Real-World GitHub Issues?cited by 3,373
- 2023AutoGen: Enabling Next-Gen LLM Applications via Multi-Agent Conversationcited by 1,988
- 2024Scaling Synthetic Data Creation with 1,000,000,000 Personascited by 417
- 2025AgentSociety: Large-Scale Simulation of LLM-Driven Generative Agents Advances Understanding of Human Behaviors and Societycited by 186
- 2025MultiAgentBench: Evaluating the Collaboration and Competition of LLM agentscited by 178
- 2023EconAgent: Large Language Model-Empowered Agents for Simulating Macroeconomic Activitiescited by 154
- 2024Simulating Classroom Education with LLM-Empowered Agentscited by 127
- 2024SciAgents: Automating scientific discovery through multi-agent intelligent graph reasoningcited by 94
- 2025CitySim: Modeling Urban Behaviors and City Dynamics with Large-Scale LLM-Driven Agent Simulationcited by 28
- 2026DevBench: A Realistic, Developer-Informed Benchmark for Code Generation Modelscited by 8
- 2025Agent-Kernel: A MicroKernel Multi-Agent System Framework for Adaptive Social Simulation Powered by LLMscited by 3
Built on by
Citing papers per year: 2026: 1.
1 paper.
Similar, not cited
Close in wording, no citation either way
- 2025A Survey of LLM-based Agents in Medicine: How far are we from Baymax?
- 2024Hippocrates: An Open-Source Framework for Advancing Large Language Models in Healthcare
- 2024AI Hospital: Benchmarking Large Language Models in a Multi-agent Medical Interaction Simulator
- 2026Agentic AI in Healthcare & Medicine: A Seven-Dimensional Taxonomy for Empirical Evaluation of LLM-based Agents
- 2023Conversational Health Agents: A Personalized LLM-Powered Agent Framework
- 2024Exploring Large Language Model based Intelligent Agents: Definitions, Methods, and Prospects
- 2023Large Language Models Illuminate a Progressive Pathway to Artificial Healthcare Assistant: A Review
- 2024Adaptive Reasoning and Acting in Medical Language Agents