mlsanityData to 7 Oct 2026

MLCopilot: Unleashing the Power of Large Language Models in Solving Machine Learning Tasks

arXiv 2304.14979 ↗First posted 28 Apr 2023cs.LG cs.AIcited by 66

Not on the map. Why?

Cited by 25 in LLM agents and tool use, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

The field of machine learning (ML) has gained widespread adoption, leading to significant demand for adapting ML to specific scenarios, which is yet expensive and non-trivial. The predominant approaches towards the automation of solving ML tasks (e.g., AutoML) are often time-consuming and hard to understand for human developers. In contrast, though human engineers have the incredible ability to understand tasks and reason about solutions, their experience and knowledge are often sparse and difficult to utilize by quantitative approaches. In this paper, we aim to bridge the gap between machine intelligence and human knowledge by introducing a novel framework, which leverages the state-of-the-art large language models to develop ML solutions for novel tasks. We showcase the possibility of extending the capability of LLMs to comprehend structured inputs and perform thorough reasoning for solving novel ML tasks. And we find that, after some dedicated design, the LLM can (i) observe from the existing experiences of ML tasks and (ii) reason effectively to deliver promising results for new tasks. The solution generated can be used directly to achieve high levels of competitiveness. Examples and code available at https://github.com/microsoft/CoML.

Key facts. Cited by 66 papers in the map (data through 7 Oct 2026). Builds on 29 of its 97 references that are in the map. First posted on arXiv 28 Apr 2023.

Builds on

29 of 97 references.

References · 29

  1. 2017Attention Is All You Needcited by 59,932
  2. 2020Language Models are Few-Shot Learnerscited by 32,199
  3. 2022Training language models to follow instructions with human feedbackcited by 15,712
  4. 2023LLaMA: Open and Efficient Foundation Language Modelscited by 13,958
  5. 2022PaLM: Scaling Language Modeling with Pathwayscited by 5,304
  6. 2016XGBoost: A Scalable Tree Boosting Systemcited by 5,120
  7. 2023A Survey of Large Language Modelscited by 2,537
  8. 2022Emergent Abilities of Large Language Modelscited by 2,483
  9. 2022Rethinking the Role of Demonstrations: What Makes In-Context Learning Work?cited by 1,413
  10. 2021Fantastically Ordered Prompts and Where to Find Them: Overcoming Few-Shot Prompt Order Sensitivitycited by 1,122
  11. 2021Are NLP Models really able to Solve Simple Math Word Problems?cited by 861
  12. 2018Efficient Neural Architecture Search via Parameter Sharingcited by 831
17 more
  1. 2018A Tutorial on Bayesian Optimizationcited by 734
  2. 2016Learning to learn by gradient descent by gradient descentcited by 616
  3. 2022Towards Reasoning in Large Language Models: A Surveycited by 593
  4. 2014OpenML: networked science in machine learningcited by 523
  5. 2019Analysing Mathematical Reasoning Abilities of Neural Modelscited by 331
  6. 2021Do Prompt-Based Models Really Understand the Meaning of their Prompts?cited by 324
  7. 2022Generate rather than Retrieve: Large Language Models are Strong Context Generatorscited by 296
  8. 2019FLAML: A Fast and Lightweight AutoML Librarycited by 104
  9. 2021Representing Numbers in NLP: a Survey and a Visioncited by 83
  10. 2022Towards Learning Universal Hyperparameter Optimizers with Transformerscited by 81
  11. 2021HPO-B: A Large-Scale Reproducible Benchmark for Black-Box HPO based on OpenMLcited by 75
  12. 2023Can GPT-4 Perform Neural Architecture Search?cited by 74
  13. 2021Pre-trained Gaussian Processes for Bayesian Optimizationcited by 63
  14. 2022DeepKE: A Deep Learning Based Knowledge Extraction Toolkit for Knowledge Base Populationcited by 17
  15. 2020HyperSTAR: Task-Aware Hyperparameters for Deep Networkscited by 9
  16. 2022LUNA: Language Understanding with Number Augmentations on Transformers via Number Plugins and Pre-trainingcited by 8
  17. 2022Privacy-preserving Online AutoML for Domain-Specific Face Detectioncited by 4

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 6; 2024: 26; 2025: 19; 2026: 15.

66 papers.

Sort:

By area:

  1. 2023MLAgentBench: Evaluating Language Agents on Machine Learning Experimentationcited by 339
  2. 2023LLMLingua: Compressing Prompts for Accelerated Inference of Large Language Modelscited by 279
  3. 2024Data Interpreter: An LLM Agent For Data Sciencecited by 222
  4. 2024DS-Agent: Automated Data Science by Empowering Large Language Models with Case-Based Reasoningcited by 119
  5. 2024C3oT: Generating Shorter Chain-of-Thought without Compromising Effectivenesscited by 117
  6. 2024AutoML-Agent: A Multi-Agent LLM Framework for Full-Pipeline AutoMLcited by 84
  7. 2024AutoKaggle: A Multi-Agent Framework for Autonomous Data Science Competitionscited by 70
  8. 2025Paper2Code: Automating Code Generation from Scientific Papers in Machine Learningcited by 67
  9. 2024Retrieval Augmented Generation (RAG) and Beyond: A Comprehensive Survey on How to Make your LLMs use External Data More Wiselycited by 65
  10. 2024Large Language Model Agent for Hyper-Parameter Optimizationcited by 62
  11. 2025ML-Master: Towards AI-for-AI via Integration of Exploration and Reasoningcited by 57
  12. 2023Large Language Models Empowered Autonomous Edge AI for Connected Intelligencecited by 55
38 more
  1. 2024VisEval: A Benchmark for Data Visualization in the Era of Large Language Modelscited by 43
  2. 2025Transforming Science with Large Language Models: A Survey on AI-assisted Scientific Discovery, Experimentation, Content Generation, and Evaluationcited by 41
  3. 2025EXP-Bench: Can AI Conduct AI Research Experiments?cited by 36
  4. 2023AutoML in the Age of Large Language Models: Current Challenges, Future Opportunities and Riskscited by 34
  5. 2025Curie: Toward Rigorous and Automated Scientific Experimentation with AI Agentscited by 31
  6. 2024Benchmarking Data Science Agentscited by 30
  7. 2024InsightBench: Evaluating Business Analytics Agents Through Multi-Step Insight Generationcited by 30
  8. 2024LAMBDA: A Large Model Based Data Agentcited by 30
  9. 2025ML-Agent: Reinforcing LLM Agents for Autonomous Machine Learning Engineeringcited by 26
  10. 2024SELA: Tree-Search Enhanced LLM Agents for Automated Machine Learningcited by 23
  11. 2025Dolphin: Moving Towards Closed-loop Auto-research through Thinking, Practice, and Feedbackcited by 23
  12. 2024A Survey on Large Language Model-based Agents for Statistics and Data Sciencecited by 22
  13. 2025nvAgent: Automated Data Visualization from Natural Language via Collaborative Agent Workflowcited by 21
  14. 2024BudgetMLAgent: A Cost-Effective LLM Multi-Agent system for Automating Machine Learning Taskscited by 19
  15. 2024Large Language Models for Constructing and Optimizing Machine Learning Workflows: A Surveycited by 19
  16. 2025AgentTTS: Large Language Model Agent for Test-time Compute-optimal Scaling Strategy in Complex Taskscited by 16
  17. 2024AutoElicit: Using Large Language Models for Expert Prior Elicitation in Predictive Modellingcited by 15
  18. 2024Large Language Models Synergize with Automated Machine Learningcited by 8
  19. 2024BioKGBench: A Knowledge Graph Checking Benchmark of AI Agent for Biomedical Sciencecited by 8
  20. 2025From Hypothesis to Publication: A Comprehensive Survey of AI-Driven Research Support Systemscited by 8
  21. 2025DatawiseAgent: A Notebook-Centric LLM Agent Framework for Adaptive and Robust Data Science Automationcited by 8
  22. 2024DataLab: A Unified Platform for LLM-Powered Business Intelligencecited by 7
  23. 2024Collaborative Evolving Strategy for Automatic Data-Centric Developmentcited by 6
  24. 2026LongDA: Benchmarking LLM Agents for Long-Document Data Analysiscited by 6
  25. 2023Towards Message Brokers for Generative AI: Survey, Challenges, and Opportunitiescited by 5
  26. 2024TransitGPT: A Generative AI-based framework for interacting with GTFS data using Large Language Modelscited by 5
  27. 2025DSMentor: Enhancing Data Science Agents with Curriculum Learning and Online Knowledge Accumulationcited by 4
  28. 2024The CAP Principle for LLM Serving: A Survey of Long-Context Large Language Model Servingcited by 3
  29. 2026DARE-bench: Evaluating Modeling and Instruction Fidelity of LLMs in Data Sciencecited by 3
  30. 2026Why Solve It Twice? Hierarchical Accumulation of Skills for Transfer-Efficient ML Engineeringcited by 3
  31. 2025Empowering AI to Generate Better AI Code: Guided Generation of Deep Learning Projects with LLMscited by 2
  32. 2025Make Still Further Progress: Chain of Thoughts for Tabular Data Leaderboardcited by 2
  33. 2025Small LLMs with Expert Blocks Are Good Enough for Hyperparamter Tuningcited by 2
  34. 2024Proficient Graph Neural Network Design by Accumulating Knowledge on Large Language Modelscited by 1
  35. 2024Game On: Towards Language Models as RL Experimenterscited by 1
  36. 2024ML Research Benchmarkcited by 1
  37. 2025LightAutoDS-Tab: Multi-AutoML Agentic System for Tabular Datacited by 1
  38. 2026From Bits to Chips: An LLM-based Hardware-Aware Quantization Agent for Streamlined Deployment of LLMscited by 1

50 most cited of 66 All 66 (JSON)