# I-MCTS: Enhancing Agentic AutoML via Introspective Monte Carlo Tree Search

arXiv: https://arxiv.org/abs/2502.14693 · HTML: https://mlsanity.com/paper/2502.14693/

## Key facts

- Cited by 14 papers in the map (data through 14 Oct 2026).
- Placed in General LLM agent frameworks and benchmarks, one of 16 directions in LLM agents and tool use.
- Builds on 10 of its 40 references that are in the map.
- First posted on arXiv 20 Feb 2025.

## Placed in

- LLM agents and tool use / [General LLM agent frameworks and benchmarks](https://mlsanity.com/agents/general-llm-agent-frameworks-and-benchmarks/)

## Builds on (most cited first, 10 of 10)

- [Self-Refine: Iterative Refinement with Self-Feedback](https://mlsanity.com/paper/2303.17651/) (2023)
- [SWE-bench: Can Language Models Resolve Real-World GitHub Issues?](https://mlsanity.com/paper/2310.06770/) (2023)
- [A-MEM: Agentic Memory for LLM Agents](https://mlsanity.com/paper/2502.12110/) (2025)
- [Language Agent Tree Search Unifies Reasoning Acting and Planning in Language Models](https://mlsanity.com/paper/2310.04406/) (2023)
- [MLE-bench: Evaluating Machine Learning Agents on Machine Learning Engineering](https://mlsanity.com/paper/2410.07095/) (2024)
- [MLAgentBench: Evaluating Language Agents on Machine Learning Experimentation](https://mlsanity.com/paper/2310.03302/) (2023)
- [Data Interpreter: An LLM Agent For Data Science](https://mlsanity.com/paper/2402.18679/) (2024)
- [Recursive Introspection: Teaching Language Model Agents How to Self-Improve](https://mlsanity.com/paper/2407.18219/) (2024)
- [AutoKaggle: A Multi-Agent Framework for Autonomous Data Science Competitions](https://mlsanity.com/paper/2410.20424/) (2024)
- [AutoGluon-Multimodal (AutoMM): Supercharging Multimodal AutoML with Foundation Models](https://mlsanity.com/paper/2404.16233/) (2024)

## Built on by (most cited first, 14 of 14)

- [MLZero: A Multi-Agent System for End-to-end Machine Learning Automation](https://mlsanity.com/paper/2505.13941/) (2025)
- [Reinforcement Learning for Machine Learning Engineering Agents](https://mlsanity.com/paper/2509.01684/) (2025)
- [MAS2: Self-Generative, Self-Configuring, Self-Rectifying Multi-Agent Systems](https://mlsanity.com/paper/2509.24323/) (2025)
- [Can Agentic AI Match the Performance of Human Data Scientists?](https://mlsanity.com/paper/2512.20959/) (2025)
- [Test-time Scaling of LLMs: A Survey from A Subproblem Structure Perspective](https://mlsanity.com/paper/2511.14772/) (2025)
- [SEA-TS: Self-Evolving Agent for Autonomous Code Generation of Time Series Forecasting Algorithms](https://mlsanity.com/paper/2603.04873/) (2026)
- [AgentDS Technical Report: Benchmarking the Future of Human-AI Collaboration in Domain-Specific Data Science](https://mlsanity.com/paper/2603.19005/) (2026)
- [PRISM-MCTS: Learning from Reasoning Trajectories with Metacognitive Reflection](https://mlsanity.com/paper/2604.05424/) (2026)
- [AgentGA: Evolving Code Solutions in Agent-Seed Space](https://mlsanity.com/paper/2604.14655/) (2026)
- [Learning the ARTS of Search for Automated Discovery](https://mlsanity.com/paper/2606.21891/) (2026)
- [A Framework for Assessing AI Agent Decisions and Outcomes in AutoML Pipelines](https://mlsanity.com/paper/2602.22442/) (2026)
- [Bridging Inference-Time Scaling and Episodic Memory with Action-Centric Graphs](https://mlsanity.com/paper/2607.27415/) (2026)
- [CEDAR: Agent-Orchestrated Tree Search for Goal-Directed Optimization of Complex Systems](https://mlsanity.com/paper/2608.06871/) (2026)
- [SkillVine: Agent Skill Evolution via Branching Exploration](https://mlsanity.com/paper/2609.32731/) (2026)

## Abstract (arXiv)

Recent advancements in large language models (LLMs) have shown remarkable potential in automating machine learning tasks. However, existing LLM-based agents often struggle with low-diversity and suboptimal code generation. While recent work has introduced Monte Carlo Tree Search (MCTS) to address these issues, limitations persist in the quality and diversity of thoughts generated, as well as in the scalar value feedback mechanisms used for node selection. In this study, we introduce Introspective Monte Carlo Tree Search (I-MCTS), a novel approach that iteratively expands tree nodes through an introspective process that meticulously analyzes solutions and results from parent and sibling nodes. This facilitates a continuous refinement of the node in the search tree, thereby enhancing the overall decision-making process. Furthermore, we integrate a Large Language Model (LLM)-based value model to facilitate direct evaluation of each node's solution prior to conducting comprehensive computational rollouts. A hybrid rewarding mechanism is implemented to seamlessly transition the Q-value from LLM-estimated scores to actual performance scores. This allows higher-quality nodes to be traversed earlier. Applied to the various ML tasks, our approach demonstrates a 4% absolute improvement in performance compared to the strong open-source AutoML agents, showcasing its effectiveness in enhancing agentic AutoML systems. Resource available at https://github.com/jokieleung/I-MCTS