mlsanityData to 14 Oct 2026

Benchmark for Planning and Control with Large Language Model Agents: Blocksworld with Model Context Protocol

arXiv 2512.03955 ↗First posted 3 Dec 2025cs.AI cs.ETcited by 3

On the map: LLM agents and tool use → General LLM agent frameworks and benchmarks

Abstract (arXiv)

Industrial automation increasingly requires flexible control strategies that can adapt to changing tasks and environments. Agents based on Large Language Models (LLMs) offer potential for such adaptive planning and execution but lack standardized benchmarks for systematic comparison. We introduce a benchmark with an executable simulation environment representing the Blocksworld problem providing five complexity categories. By integrating the Model Context Protocol (MCP) as a standardized tool interface, diverse agent architectures can be connected to and evaluated against the benchmark without implementation-specific modifications. A single-agent implementation demonstrates the benchmark's applicability, establishing quantitative metrics for comparison of LLM-based planning and execution approaches.

Key facts. Cited by 3 papers in the map (data through 14 Oct 2026). Placed in General LLM agent frameworks and benchmarks, one of 16 directions in LLM agents and tool use. Builds on 11 of its 17 references that are in the map. First posted on arXiv 3 Dec 2025.

Builds on

11 of 17 references.

Closest ancestors · 4

Same direction, newest first

  1. 2025PLANET: A Collection of Benchmarks for Evaluating LLMs' Planning Capabilities21 Apr
  2. 2023A Survey on Large Language Model based Autonomous Agents22 Aug
  3. 2023AgentBench: Evaluating LLMs as Agents7 Aug
  4. 2023Towards autonomous system: flexible modular production system enhanced with large language model agents28 Apr

Other references · 7

  1. 2022ReAct: Synergizing Reasoning and Acting in Language Modelscited by 8,695
  2. 2022PlanBench: An Extensible Benchmark for Evaluating Large Language Models on Planning and Reasoning about Changecited by 382
  3. 2025MCP-Bench: Benchmarking Tool-Using LLM Agents with Complex Real-World Tasks via MCP Serverscited by 109
  4. 2025MCP-Universe: Benchmarking Large Language Models with Real-World Model Context Protocol Serverscited by 96
  5. 2025MCP-RADAR: A Multi-Dimensional Benchmark for Evaluating Tool Use Capabilities in Large Language Modelscited by 48
  6. 2025Inference-Time Computations for LLM Reasoning and Planning: A Benchmark and Insightscited by 30
  7. 2025Beyond Formal Semantics for Capabilities and Skills: Model Context Protocol in Manufacturingcited by 4

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 3.

3 papers.

  1. 2026Toward Efficient Agents: Memory, Tool learning, and Planningcited by 16
  2. 2026From Detection to Action: Using LLM Agents for Fault-Tolerant Controlcited by 2
  3. 2026Analysis of Optimality of Large Language Models on Planning Problemscited by 1

Similar, not cited

Close in wording, no citation either way

  1. 2026Agentic LLM Planning via Step-Wise PDDL Simulation: An Empirical Characterisation
  2. 2025Large Language Models for Planning: A Comprehensive and Systematic Survey
  3. 2025Robotouille: An Asynchronous Planning Benchmark for LLM Agents
  4. 2024Incorporating Large Language Models into Production Systems for Enhanced Task Automation and Flexibility
  5. 2025Can LLM-Reasoning Models Replace Classical Planning? A Benchmark Study
  6. 2024Large Language Model based Multi-Agents: A Survey of Progress and Challenges
  7. 2025Adaptive Domain Modeling with Language Models: A Multi-Agent Approach to Task Planning
  8. 2025A Comprehensive Survey on Benchmarks and Solutions in Software Engineering of LLM-Empowered Agentic System