# AutoDS: Towards Human-Centered Automation of Data Science

arXiv: https://arxiv.org/abs/2101.05273 · HTML: https://mlsanity.com/paper/2101.05273/

## Key facts

- Cited by 32 papers in the map (data through 14 Oct 2026).
- Builds on 13 of its 67 references that are in the map.
- First posted on arXiv 13 Jan 2021.

## Builds on (most cited first, 13 of 13)

- [AMC: AutoML for Model Compression and Acceleration on Mobile Devices](https://mlsanity.com/paper/1802.03494/) (2018)
- [Metrics for Explainable AI: Challenges and Prospects](https://mlsanity.com/paper/1812.04608/) (2018)
- [Human-AI Collaboration in Data Science: Exploring Data Scientists' Perceptions of Automated AI](https://mlsanity.com/paper/1909.02309/) (2019)
- [How do Data Science Workers Collaborate? Roles, Workflows, and Tools](https://mlsanity.com/paper/2001.06684/) (2020)
- [An ADMM Based Framework for AutoML Pipeline Configuration](https://mlsanity.com/paper/1905.00424/) (2019)
- [One button machine for automating feature engineering in relational databases](https://mlsanity.com/paper/1706.00327/) (2017)
- [How Data Scientists Work Together With Domain Experts in Scientific Collaborations: To Find The Right Answer Or To Ask The Right Question?](https://mlsanity.com/paper/1909.03486/) (2019)
- [Calendar.help: Designing a Workflow-Based Scheduling Agent with Humans in the Loop](https://mlsanity.com/paper/1703.08428/) (2017)
- [AutoAIViz: Opening the Blackbox of Automated Artificial Intelligence with Conditional Parallel Coordinates](https://mlsanity.com/paper/1912.06723/) (2019)
- [ATMSeer: Increasing Transparency and Controllability in Automated Machine Learning](https://mlsanity.com/paper/1902.05009/) (2019)
- [How Much Automation Does a Data Scientist Want?](https://mlsanity.com/paper/2101.03970/) (2021)
- [Benchmark and Survey of Automated Machine Learning Frameworks](https://mlsanity.com/paper/1904.12054/) (2019)
- [Multi-Armed Bandits with Fairness Constraints for Distributing Resources to Human Teammates](https://mlsanity.com/paper/1907.00313/) (2019)

## Built on by (most cited first, 30 of 32)

- [Natural Language to Code Generation in Interactive Data Science Notebooks](https://mlsanity.com/paper/2212.09248/) (2022)
- [DA-Code: Agent Data Science Code Generation Benchmark for Large Language Models](https://mlsanity.com/paper/2410.07331/) (2024)
- [Improving Steering and Verification in AI-Assisted Data Analysis with Interactive Task Decomposition](https://mlsanity.com/paper/2407.02651/) (2024)
- [Reinforcement-Enhanced Autoregressive Feature Transformation: Gradient-steered Search in Continuous Space for Postfix Expressions](https://mlsanity.com/paper/2309.13618/) (2023)
- [Model Compression in Practice: Lessons Learned from Practitioners Creating On-device Machine Learning Experiences](https://mlsanity.com/paper/2310.04621/) (2023)
- [Unsupervised Generative Feature Transformation via Graph Contrastive Pre-training and Multi-objective Fine-tuning](https://mlsanity.com/paper/2405.16879/) (2024)
- [Fast and Informative Model Selection using Learning Curve Cross-Validation](https://mlsanity.com/paper/2111.13914/) (2021)
- [Telling Stories from Computational Notebooks: AI-Assisted Presentation Slides Creation for Presenting Data Science Work](https://mlsanity.com/paper/2203.11085/) (2022)
- [Model LineUpper: Supporting Interactive Model Comparison at Multiple Levels for AutoML](https://mlsanity.com/paper/2104.04375/) (2021)
- [Competent but Rigid: Identifying the Gap in Empowering AI to Participate Equally in Group Decision-Making](https://mlsanity.com/paper/2302.08807/) (2023)
- [XAutoML: A Visual Analytics Tool for Understanding and Validating Automated Machine Learning](https://mlsanity.com/paper/2202.11954/) (2022)
- [AutoML in The Wild: Obstacles, Workarounds, and Expectations](https://mlsanity.com/paper/2302.10827/) (2023)
- [SPIO: Ensemble and Selective Strategies via LLM-Based Multi-Agent Planning in Automated Data Science](https://mlsanity.com/paper/2503.23314/) (2025)
- [Enhancing Tabular Data Optimization with a Flexible Graph-based Reinforced Exploration Strategy](https://mlsanity.com/paper/2406.07404/) (2024)
- [FastFT: Accelerating Reinforced Feature Transformation via Advanced Exploration Strategies](https://mlsanity.com/paper/2503.20394/) (2025)
- [Efficient Human-in-the-loop System for Guiding DNNs Attention](https://mlsanity.com/paper/2206.05981/) (2022)
- [Visual Analysis of Neural Architecture Spaces for Summarizing Design Principles](https://mlsanity.com/paper/2208.09665/) (2022)
- [Human-Centered AI for Data Science: A Systematic Approach](https://mlsanity.com/paper/2110.01108/) (2021)
- [OmniForce: On Human-Centered, Large Model Empowered and Cloud-Edge Collaborative AutoML System](https://mlsanity.com/paper/2303.00501/) (2023)
- [Collaborative Multi-Agent Reinforcement Learning for Automated Feature Transformation with Graph-Driven Path Optimization](https://mlsanity.com/paper/2504.17355/) (2025)
- [What Makes LLM Agent Simulations Useful for Policy Practice? An Iterative Design Study in Emergency Preparedness](https://mlsanity.com/paper/2509.21868/) (2025)
- [Hierarchical and Permutation-Invariant Feature Transformation Learning via Policy-Guided Embedding Search](https://mlsanity.com/paper/2609.10225/) (2026)
- [Naive Automated Machine Learning -- A Late Baseline for AutoML](https://mlsanity.com/paper/2103.10496/) (2021)
- [Practitioner Motives to Use Different Hyperparameter Optimization Methods](https://mlsanity.com/paper/2203.01717/) (2022)
- [Evolving machine learning workflows through interactive AutoML](https://mlsanity.com/paper/2402.18505/) (2024)
- [Assessment of cognitive characteristics in intelligent systems and predictive ability](https://mlsanity.com/paper/2209.11761/) (2022)
- [Human-Centered AI Product Prototyping with No-Code AutoML: Conceptual Framework, Potentials and Limitations](https://mlsanity.com/paper/2402.07933/) (2024)
- [Trust Your Gut: Comparing Human and Machine Inference from Noisy Visualizations](https://mlsanity.com/paper/2407.16871/) (2024)
- [How to Measure Human-AI Prediction Accuracy in Explainable AI Systems](https://mlsanity.com/paper/2409.00069/) (2024)
- [DeepFeature: LLM-Empowered Context-aware Feature Generation for Wearable Biosignals](https://mlsanity.com/paper/2512.08379/) (2025)

## Abstract (arXiv)

Data science (DS) projects often follow a lifecycle that consists of laborious tasks for data scientists and domain experts (e.g., data exploration, model training, etc.). Only till recently, machine learning(ML) researchers have developed promising automation techniques to aid data workers in these tasks. This paper introduces AutoDS, an automated machine learning (AutoML) system that aims to leverage the latest ML automation techniques to support data science projects. Data workers only need to upload their dataset, then the system can automatically suggest ML configurations, preprocess data, select algorithm, and train the model. These suggestions are presented to the user via a web-based graphical user interface and a notebook-based programming user interface. We studied AutoDS with 30 professional data scientists, where one group used AutoDS, and the other did not, to complete a data science project. As expected, AutoDS improves productivity; Yet surprisingly, we find that the models produced by the AutoDS group have higher quality and less errors, but lower human confidence scores. We reflect on the findings by presenting design implications for incorporating automation techniques into human work in the data science lifecycle.