# Data Engineering for Everyone

arXiv: https://arxiv.org/abs/2102.11447 · HTML: https://mlsanity.com/paper/2102.11447/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- Builds on 0 of its 11 references that are in the map.
- First posted on arXiv 23 Feb 2021.

## Built on by (most cited first, 5 of 5)

- [Training Data Influence Analysis and Estimation: A Survey](https://mlsanity.com/paper/2212.04612/) (2022)
- [The People's Speech: A Large-Scale Diverse English Speech Recognition Dataset for Commercial Usage](https://mlsanity.com/paper/2111.09344/) (2021)
- [Machine Learning Sensors](https://mlsanity.com/paper/2206.03266/) (2022)
- [Fast-DataShapley: Neural Modeling for Training Data Valuation](https://mlsanity.com/paper/2506.05281/) (2025)
- [Targeted Fine-Tuning of DNN-Based Receivers via Influence Functions](https://mlsanity.com/paper/2509.15950/) (2025)

## Abstract (arXiv)

Data engineering is one of the fastest-growing fields within machine learning (ML). As ML becomes more common, the appetite for data grows more ravenous. But ML requires more data than individual teams of data engineers can readily produce, which presents a severe challenge to ML deployment at scale. Much like the software-engineering revolution, where mass adoption of open-source software replaced the closed, in-house development model for infrastructure code, there is a growing need to enable rapid development and open contribution to massive machine learning data sets. This article shows that open-source data sets are the rocket fuel for research and innovation at even some of the largest AI organizations. Our analysis of nearly 2000 research publications from Facebook, Google and Microsoft over the past five years shows the widespread use and adoption of open data sets. Open data sets that are easily accessible to the public are vital to accelerating ML innovation for everyone. But such open resources are scarce in the wild. So, what if we are able to accelerate data-set creation via automatic data set generation tools?