# Data Engineering for Data Analytics: A Classification of the Issues, and Case Studies

arXiv: https://arxiv.org/abs/2004.12929 · HTML: https://mlsanity.com/paper/2004.12929/

## Key facts

- Cited by 10 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 27 Apr 2020.

## Built on by (most cited first, 10 of 10)

- [Natural Language to Code Generation in Interactive Data Science Notebooks](https://mlsanity.com/paper/2212.09248/) (2022)
- [Automating Data Science: Prospects and Challenges](https://mlsanity.com/paper/2105.05699/) (2021)
- [An Empirical Evaluation of Flow Based Programming in the Machine Learning Deployment Context](https://mlsanity.com/paper/2204.12781/) (2022)
- [Towards better data discovery and collection with flow-based programming](https://mlsanity.com/paper/2108.04105/) (2021)
- [Machine Learning Systems: A Survey from a Data-Oriented Perspective](https://mlsanity.com/paper/2302.04810/) (2023)
- [A Survey of Pipeline Tools for Data Engineering](https://mlsanity.com/paper/2406.08335/) (2024)
- [Black Box Absorption: LLMs Undermining Innovative Ideas](https://mlsanity.com/paper/2510.20612/) (2025)
- [Empowering the trustworthiness of ML-based critical systems through engineering activities](https://mlsanity.com/paper/2209.15438/) (2022)
- [Being Accountable is Smart: Navigating the Technical and Regulatory Landscape of AI-based Services for Power Grid](https://mlsanity.com/paper/2408.01121/) (2024)
- [Robustness of AutoML on Dirty Categorical Data](https://mlsanity.com/paper/2602.00412/) (2026)

## Abstract (arXiv)

Consider the situation where a data analyst wishes to carry out an analysis on a given dataset. It is widely recognized that most of the analyst's time will be taken up with data engineering tasks such as acquiring, understanding, cleaning and preparing the data. In this paper we provide a description and classification of such tasks into high-levels groups, namely data organization, data quality and feature engineering. We also make available four datasets and example analyses that exhibit a wide variety of these problems, to help encourage the development of tools and techniques to help reduce this burden and push forward research towards the automation or semi-automation of the data engineering process.