mlsanityData to 14 Oct 2026

DS-Agent: Automated Data Science by Empowering Large Language Models with Case-Based Reasoning

arXiv 2402.17453 ↗First posted 27 Feb 2024cs.LGcited by 125

On the map: LLM reasoning → Reasoning for agents, planning and code

also in 1 more
Abstract (arXiv)

In this work, we investigate the potential of large language models (LLMs) based agents to automate data science tasks, with the goal of comprehending task requirements, then building and training the best-fit machine learning models. Despite their widespread success, existing LLM agents are hindered by generating unreasonable experiment plans within this scenario. To this end, we present DS-Agent, a novel automatic framework that harnesses LLM agent and case-based reasoning (CBR). In the development stage, DS-Agent follows the CBR framework to structure an automatic iteration pipeline, which can flexibly capitalize on the expert knowledge from Kaggle, and facilitate consistent performance improvement through the feedback mechanism. Moreover, DS-Agent implements a low-resource deployment stage with a simplified CBR paradigm to adapt past successful solutions from the development stage for direct code generation, significantly reducing the demand on foundational capabilities of LLMs. Empirically, DS-Agent with GPT-4 achieves 100% success rate in the development stage, while attaining 36% improvement on average one pass rate across alternative LLMs in the deployment stage. In both stages, DS-Agent achieves the best rank in performance, costing 1.60 and 0.13 per run with GPT-4, respectively. Our data and code are open-sourced at https://github.com/guosyjlu/DS-Agent.

Key facts. Cited by 125 papers in the map (data through 14 Oct 2026). Placed in Reasoning for agents, planning and code, one of 17 directions in LLM reasoning. Builds on 31 of its 68 references that are in the map. First posted on arXiv 27 Feb 2024.

Builds on

31 of 68 references.

Closest ancestors · 1

Same direction, newest first

  1. 2023Pangu-Agent: A Fine-Tunable Generalist Agent with Structured Reasoning22 Dec

Other references · 30

  1. 2023GPT-4 Technical Reportcited by 18,396
  2. 2020Retrieval-Augmented Generation for Knowledge-Intensive NLP Taskscited by 9,297
  3. 2022ReAct: Synergizing Reasoning and Acting in Language Modelscited by 8,695
  4. 2023Efficient Memory Management for Large Language Model Serving with PagedAttentioncited by 5,944
  5. 2023Reflexion: Language Agents with Verbal Reinforcement Learningcited by 4,276
  6. 2022Do As I Can, Not As I Say: Grounding Language in Robotic Affordancescited by 2,348
  7. 2023Retrieval-Augmented Generation for Large Language Models: A Surveycited by 2,212
  8. 2024Mixtral of Expertscited by 1,823
  9. 2023MetaGPT: Meta Programming for A Multi-Agent Collaborative Frameworkcited by 1,808
  10. 2023HuggingGPT: Solving AI Tasks with ChatGPT and its Friends in Hugging Facecited by 1,190
  11. 2023ExpeL: LLM Agents Are Experiential Learnerscited by 876
  12. 2023REPLUG: Retrieval-Augmented Black-Box Language Modelscited by 792
18 more
  1. 2021Learning To Retrieve Prompts for In-Context Learningcited by 648
  2. 2022DS-1000: A Natural and Reliable Benchmark for Data Science Code Generationcited by 452
  3. 2020AutoGluon-Tabular: Robust and Accurate AutoML for Structured Datacited by 383
  4. 2023Language Models can Solve Computer Taskscited by 372
  5. 2023Is ChatGPT Good at Search? Investigating Large Language Models as Re-Ranking Agentscited by 360
  6. 2023AgentTuning: Enabling Generalized Agent Abilities for LLMscited by 338
  7. 2023FireAct: Toward Language Agent Fine-tuningcited by 256
  8. 2023Grounding Large Language Models in Interactive Environments with Online Reinforcement Learningcited by 195
  9. 2023Large Language Models for Automated Data Science: Introducing CAAFE for Context-Aware Automated Feature Engineeringcited by 127
  10. 2023AutoML-GPT: Automatic Machine Learning with GPTcited by 81
  11. 2023K2: A Foundation Language Model for Geoscience Knowledge Understanding and Utilizationcited by 69
  12. 2023MLCopilot: Unleashing the Power of Large Language Models in Solving Machine Learning Taskscited by 66
  13. 2020AutoML to Date and Beyond: Challenges and Opportunitiescited by 63
  14. 2023Learning to Retrieve In-Context Examples for Large Language Modelscited by 60
  15. 2023GeoGalactica: A Scientific Large Language Model in Geosciencecited by 33
  16. 2023ChatGPT as your Personal Data Scientistcited by 22
  17. 2021Automating Data Science: Prospects and Challengescited by 18
  18. 2023Case-Based Reasoning with Language Models for Classification of Logical Fallaciescited by 8

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2024: 19; 2025: 46; 2026: 60.

125 papers.

Sort:

By area:

  1. 2025Agent Laboratory: Using LLM Agents as Research Assistantscited by 425
  2. 2024Can LLMs Generate Novel Research Ideas? A Large-Scale Human Study with 100+ NLP Researcherscited by 363
  3. 2024Agent-as-a-Judge: Evaluate Agents with Agentscited by 238
  4. 2024Data Interpreter: An LLM Agent For Data Sciencecited by 225
  5. 2025Large Language Model Agent: A Survey on Methodology, Applications and Challengescited by 205
  6. 2024RE-Bench: Evaluating frontier AI R&D capabilities of language model agents against human expertscited by 175
  7. 2025Deep Research Agents: A Systematic Examination And Roadmapcited by 128
  8. 2025Memento: Fine-tuning LLM Agents without Fine-tuning LLMscited by 127
  9. 2025Multi-Agent Design: Optimizing Agents with Better Prompts and Topologiescited by 113
  10. 2024AutoML-Agent: A Multi-Agent LLM Framework for Full-Pipeline AutoMLcited by 87
  11. 2024BLADE: Benchmarking Language Model Agents for Data-Driven Sciencecited by 77
  12. 2025From AI for Science to Agentic Science: A Survey on Autonomous Scientific Discoverycited by 77
38 more
  1. 2025AgentRxiv: Towards Collaborative Autonomous Researchcited by 72
  2. 2024AutoKaggle: A Multi-Agent Framework for Autonomous Data Science Competitionscited by 70
  3. 2026Memento-Skills: Let Agents Design Agentscited by 62
  4. 2025MLE-STAR: Machine Learning Engineering Agent via Search and Targeted Refinementcited by 58
  5. 2025DeepAnalyze: Agentic Large Language Models for Autonomous Data Sciencecited by 50
  6. 2025Agentic Web: Weaving the Next Web with AI Agentscited by 45
  7. 2025MLE-Dojo: Interactive Environments for Empowering LLM Agents in Machine Learning Engineeringcited by 41
  8. 2024OSCAR: Operating System Control via State-Aware Reasoning and Re-Planningcited by 38
  9. 2025HypoBench: Towards Systematic and Principled Benchmarking for Hypothesis Generationcited by 37
  10. 2025EXP-Bench: Can AI Conduct AI Research Experiments?cited by 37
  11. 2025KramaBench: A Benchmark for AI Systems on Data-to-Insight Pipelines over Data Lakescited by 33
  12. 2025SciSciGPT: Advancing Human-AI Collaboration in the Science of Sciencecited by 27
  13. 2024Aviary: training language agents on challenging scientific taskscited by 26
  14. 2024SELA: Tree-Search Enhanced LLM Agents for Automated Machine Learningcited by 24
  15. 2025MLZero: A Multi-Agent System for End-to-end Machine Learning Automationcited by 24
  16. 2025MedAgentGym: A Scalable Agentic Training Environment for Code-Centric Reasoning in Biomedical Data Sciencecited by 23
  17. 2025Large Language Model-based Data Science Agent: A Surveycited by 23
  18. 2024A Survey on Large Language Model-based Agents for Statistics and Data Sciencecited by 22
  19. 2024Agentic Information Retrievalcited by 21
  20. 2024Transforming Wearable Data into Personal Health Insights using Large Language Model Agentscited by 19
  21. 2024Large Language Models for Constructing and Optimizing Machine Learning Workflows: A Surveycited by 19
  22. 2025AutoMLGen: Navigating Fine-Grained Optimization for Coding Agentscited by 18
  23. 2026A Comprehensive Survey on Agent Skills: Taxonomy, Techniques, and Applicationscited by 18
  24. 2026Autodata: An agentic data scientist to create high quality synthetic datacited by 16
  25. 2026EvolveRouter: Co-Evolving Routing and Prompt for Multi-Agent Question Answeringcited by 15
  26. 2025Review of Case-Based Reasoning for LLM Agents: Theoretical Foundations, Architectural Components, and Cognitive Integrationcited by 12
  27. 2025Scaling Generalist Data-Analytic Agentscited by 11
  28. 2025A Survey of AI Scientistscited by 11
  29. 2026AI for Auto-Research: Roadmap & User Guidecited by 11
  30. 2024Cognitive Kernel: An Open-source Agent System towards Generalist Autopilotscited by 10
  31. 2025LLM-REVal: Can We Trust LLM Reviewers Yet?cited by 9
  32. 2026Autonomous Research Agents: A Survey of AI Scientists and the Verification Gapcited by 9
  33. 2024Automating Traffic Model Enhancement with AI Research Agentcited by 8
  34. 2024Watson: A Cognitive Observability Framework for the Reasoning of LLM-Powered Agentscited by 8
  35. 2025Agentic Knowledgeable Self-awarenesscited by 8
  36. 2025BioDSA-1K: Benchmarking Data Science Agents for Biomedical Researchcited by 8
  37. 2025LLM Meeting Decision Trees on Tabular Datacited by 8
  38. 2025SimuGen: Multi-modal Agentic Framework for Constructing Block Diagram-Based Simulation Modelscited by 8

50 most cited of 125 All 125 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2025LLM-Based Data Science Agents: A Survey of Capabilities, Challenges, and Future Directions
  2. 2025AutoMind: Adaptive Knowledgeable Agent for Automated Data Science
  3. 2025DS-STAR: Data Science Agent for Solving Diverse Tasks across Heterogeneous Formats and Open-Ended Queries
  4. 2025Measuring Data Science Automation: A Survey of Evaluation Tools for AI Assistants and Agents
  5. 2024Benchmarking Data Science Agents
  6. 2026DS-Lighting: Making Agent Harnesses Explicit for Data-Science Automation
  7. 2025DatawiseAgent: A Notebook-Centric LLM Agent Framework for Adaptive and Robust Data Science Automation
  8. 2025VDSAgents: A PCS-Guided Multi-Agent System for Veridical Data Science Automation