# An Overview of the Data-Loader Landscape: Comparative Performance Analysis

arXiv: https://arxiv.org/abs/2209.13705 · HTML: https://mlsanity.com/paper/2209.13705/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 25 of its 52 references that are in the map.
- First posted on arXiv 27 Sep 2022.

## Builds on (most cited first, 25 of 25)

- [PyTorch: An Imperative Style, High-Performance Deep Learning Library](https://mlsanity.com/paper/1912.01703/) (2019)
- [Microsoft COCO: Common Objects in Context](https://mlsanity.com/paper/1405.0312/) (2014)
- [Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks](https://mlsanity.com/paper/1506.01497/) (2015)
- [Scaling Laws for Neural Language Models](https://mlsanity.com/paper/2001.08361/) (2020)
- [Google's Neural Machine Translation System: Bridging the Gap between Human and Machine Translation](https://mlsanity.com/paper/1609.08144/) (2016)
- [Albumentations: fast and flexible image augmentations](https://mlsanity.com/paper/1809.06839/) (2018)
- [Demystifying Parallel and Distributed Deep Learning: An In-Depth Concurrency Analysis](https://mlsanity.com/paper/1802.09941/) (2018)
- [Scaling Deep Contrastive Learning Batch Size under Memory Limited Setup](https://mlsanity.com/paper/2101.06983/) (2021)
- [MLPerf Training Benchmark](https://mlsanity.com/paper/1910.01500/) (2019)
- [PyTorch Distributed: Experiences on Accelerating Data Parallel Training](https://mlsanity.com/paper/2006.15704/) (2020)
- [Benchmark Analysis of Representative Deep Neural Network Architectures](https://mlsanity.com/paper/1810.00736/) (2018)
- [AI Benchmark: Running Deep Neural Networks on Android Smartphones](https://mlsanity.com/paper/1810.01109/) (2018)
- [Analyzing and Mitigating Data Stalls in DNN Training](https://mlsanity.com/paper/2007.06775/) (2020)
- [Analysis of DAWNBench, a Time-to-Accuracy Machine Learning Performance Benchmark](https://mlsanity.com/paper/1806.01427/) (2018)
- [Fathom: Reference Workloads for Modern Deep Learning Methods](https://mlsanity.com/paper/1608.06581/) (2016)
- [Benchmarking State-of-the-Art Deep Learning Software Tools](https://mlsanity.com/paper/1608.07249/) (2016)
- [Plumber: Diagnosing and Removing Performance Bottlenecks in Machine Learning Data Pipelines](https://mlsanity.com/paper/2111.04131/) (2021)
- [Deep Lake: a Lakehouse for Deep Learning](https://mlsanity.com/paper/2209.10785/) (2022)
- [dPRO: A Generic Profiling and Optimization System for Expediting Distributed DNN Training](https://mlsanity.com/paper/2205.02473/) (2022)
- [A Comparative Measurement Study of Deep Learning as a Service Framework](https://mlsanity.com/paper/1810.12210/) (2018)
- [A Modular Benchmarking Infrastructure for High-Performance and Reproducible Deep Learning](https://mlsanity.com/paper/1901.10183/) (2019)
- [TBD: Benchmarking and Analyzing Deep Neural Network Training](https://mlsanity.com/paper/1803.06905/) (2018)
- [AI Matrix: A Deep Learning Benchmark for Alibaba Data Centers](https://mlsanity.com/paper/1909.10562/) (2019)
- [BENCHIP: Benchmarking Intelligence Processors](https://mlsanity.com/paper/1710.08315/) (2017)
- [AIBench: An Agile Domain-specific Benchmarking Methodology and an AI Benchmark Suite](https://mlsanity.com/paper/2002.07162/) (2020)

## Built on by (most cited first, 1 of 1)

- [A Selective Review on Statistical Methods for Massive Data Computation: Distributed Computing, Subsampling, and Minibatch Techniques](https://mlsanity.com/paper/2403.11163/) (2024)

## Abstract (arXiv)

Dataloaders, in charge of moving data from storage into GPUs while training machine learning models, might hold the key to drastically improving the performance of training jobs. Recent advances have shown promise not only by considerably decreasing training time but also by offering new features such as loading data from remote storage like S3. In this paper, we are the first to distinguish the dataloader as a separate component in the Deep Learning (DL) workflow and to outline its structure and features. Finally, we offer a comprehensive comparison of the different dataloading libraries available, their trade-offs in terms of functionality, usability, and performance and the insights derived from them.