# SPIO: Ensemble and Selective Strategies via LLM-Based Multi-Agent Planning in Automated Data Science arXiv: https://arxiv.org/abs/2503.23314 ยท HTML: https://mlsanity.com/paper/2503.23314/ ## Key facts - Cited by 7 papers in the map (data through 14 Oct 2026). - Placed in Reasoning for agents, planning and code, one of 17 directions in LLM reasoning. - Builds on 38 of its 60 references that are in the map. - First posted on arXiv 30 Mar 2025. ## Placed in - LLM reasoning / [Reasoning for agents, planning and code](https://mlsanity.com/lr/reasoning-for-agents-planning-and-code/) - LLM agents and tool use / [Web, search, and research agents](https://mlsanity.com/agents/web-search-and-research-agents/) ## Builds on (most cited first, 30 of 38) - [GPT-4 Technical Report](https://mlsanity.com/paper/2303.08774/) (2023) - [OpenHands: An Open Platform for AI Software Developers as Generalist Agents](https://mlsanity.com/paper/2407.16741/) (2024) - [CodeGen: An Open Large Language Model for Code with Multi-Turn Program Synthesis](https://mlsanity.com/paper/2203.13474/) (2022) - [Unsupervised Deep Embedding for Clustering Analysis](https://mlsanity.com/paper/1511.06335/) (2015) - [Executable Code Actions Elicit Better LLM Agents](https://mlsanity.com/paper/2402.01030/) (2024) - [OpenML: networked science in machine learning](https://mlsanity.com/paper/1407.7722/) (2014) - [AutoGluon-Tabular: Robust and Accurate AutoML for Structured Data](https://mlsanity.com/paper/2003.06505/) (2020) - [MLAgentBench: Evaluating Language Agents on Machine Learning Experimentation](https://mlsanity.com/paper/2310.03302/) (2023) - [AIDE: AI-Driven Exploration in the Space of Code](https://mlsanity.com/paper/2502.13138/) (2025) - [Data Interpreter: An LLM Agent For Data Science](https://mlsanity.com/paper/2402.18679/) (2024) - [Auto-Sklearn 2.0: Hands-free AutoML via Meta-Learning](https://mlsanity.com/paper/2007.04074/) (2020) - [OpenAgents: An Open Platform for Language Agents in the Wild](https://mlsanity.com/paper/2310.10634/) (2023) - [Large Language Models for Automated Data Science: Introducing CAAFE for Context-Aware Automated Feature Engineering](https://mlsanity.com/paper/2305.03403/) (2023) - [DS-Agent: Automated Data Science by Empowering Large Language Models with Case-Based Reasoning](https://mlsanity.com/paper/2402.17453/) (2024) - [LIDA: A Tool for Automatic Generation of Grammar-Agnostic Visualizations and Infographics using Large Language Models](https://mlsanity.com/paper/2303.02927/) (2023) - [AutoML-GPT: Automatic Machine Learning with GPT](https://mlsanity.com/paper/2305.02499/) (2023) - [Is GPT-4 a Good Data Analyst?](https://mlsanity.com/paper/2305.15038/) (2023) - [AutoKaggle: A Multi-Agent Framework for Autonomous Data Science Competitions](https://mlsanity.com/paper/2410.20424/) (2024) - [A Survey on LLM-based Code Generation for Low-Resource and Domain-Specific Programming Languages](https://mlsanity.com/paper/2410.03981/) (2024) - [ML-Bench: Evaluating Large Language Models and Agents for Machine Learning Tasks on Repository-Level Code](https://mlsanity.com/paper/2311.09835/) (2023) - [Automated Machine Learning: From Principles to Practices](https://mlsanity.com/paper/1810.13306/) (2018) - [Spider2-V: How Far Are Multimodal Agents From Automating Data Science and Engineering Workflows?](https://mlsanity.com/paper/2407.10956/) (2024) - [Simple And Efficient Architecture Search for Convolutional Neural Networks](https://mlsanity.com/paper/1711.04528/) (2017) - [One button machine for automating feature engineering in relational databases](https://mlsanity.com/paper/1706.00327/) (2017) - [AutoDS: Towards Human-Centered Automation of Data Science](https://mlsanity.com/paper/2101.05273/) (2021) - [AlphaD3M: Machine Learning Pipeline Synthesis](https://mlsanity.com/paper/2111.02508/) (2021) - [Large Language Models Streamline Automated Machine Learning for Clinical Studies](https://mlsanity.com/paper/2308.14120/) (2023) - [ChatGPT as your Personal Data Scientist](https://mlsanity.com/paper/2305.13657/) (2023) - [Large Language Models for Constructing and Optimizing Machine Learning Workflows: A Survey](https://mlsanity.com/paper/2411.10478/) (2024) - [KeystoneML: Optimizing Pipelines for Large-Scale Advanced Analytics](https://mlsanity.com/paper/1610.09451/) (2016) ## Built on by (most cited first, 7 of 7) - [Large Language Model-based Data Science Agent: A Survey](https://mlsanity.com/paper/2508.02744/) (2025) - [Agent-SAMA: State-Aware Mobile Assistant](https://mlsanity.com/paper/2505.23596/) (2025) - [AssistedDS: Benchmarking How External Domain Knowledge Assists LLMs in Automated Data Science](https://mlsanity.com/paper/2506.13992/) (2025) - [AgentDS Technical Report: Benchmarking the Future of Human-AI Collaboration in Domain-Specific Data Science](https://mlsanity.com/paper/2603.19005/) (2026) - [Spec-Driven AI for Science: The ARIA Framework for Automated and Reproducible Data Analysis](https://mlsanity.com/paper/2510.11143/) (2025) - [More Agents Improve Math Problem Solving but Adversarial Robustness Gap Persists](https://mlsanity.com/paper/2511.07112/) (2025) - [CIPHER: A Decoupled Exploration-Selection Framework for Test-Time Scaling of Data Science Agents](https://mlsanity.com/paper/2607.14386/) (2026) ## Abstract (arXiv) Large Language Models (LLMs) have enabled dynamic reasoning in automated data analytics, yet recent multi-agent systems remain limited by rigid, single-path workflows that restrict strategic exploration and often lead to suboptimal outcomes. To overcome these limitations, we propose SPIO (Sequential Plan Integration and Optimization), a framework that replaces rigid workflows with adaptive, multi-path planning across four core modules: data preprocessing, feature engineering, model selection, and hyperparameter tuning. In each module, specialized agents generate diverse candidate strategies, which are cascaded and refined by an optimization agent. SPIO offers two operating modes: SPIO-S for selecting a single optimal pipeline, and SPIO-E for ensembling top-k pipelines to maximize robustness. Extensive evaluations on Kaggle and OpenML benchmarks show that SPIO consistently outperforms state-of-the-art baselines, achieving an average performance gain of 5.6%. By explicitly exploring and integrating multiple solution paths, SPIO delivers a more flexible, accurate, and reliable foundation for automated data science.