# scDataset: Scalable Data Loading for Deep Learning on Large-Scale Single-Cell Omics

arXiv: https://arxiv.org/abs/2506.01883 · HTML: https://mlsanity.com/paper/2506.01883/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 7 of its 30 references that are in the map.
- First posted on arXiv 2 Jun 2025.

## Builds on (most cited first, 7 of 7)

- [Overcoming catastrophic forgetting in neural networks](https://mlsanity.com/paper/1612.00796/) (2016)
- [On Large-Batch Training for Deep Learning: Generalization Gap and Sharp Minima](https://mlsanity.com/paper/1609.04836/) (2016)
- [Datasets: A Community Library for Natural Language Processing](https://mlsanity.com/paper/2109.02846/) (2021)
- [Ray: A Distributed Framework for Emerging AI Applications](https://mlsanity.com/paper/1712.05889/) (2017)
- [How to Build the Virtual Cell with Artificial Intelligence: Priorities and Opportunities](https://mlsanity.com/paper/2409.11654/) (2024)
- [High Performance I/O For Large Scale Deep Learning](https://mlsanity.com/paper/2001.01858/) (2020)
- [BioNeMo Framework: a modular, high-performance library for AI model development in drug discovery](https://mlsanity.com/paper/2411.10548/) (2024)

## Built on by (most cited first, 1 of 1)

- [annbatch unlocks terabyte-scale training of biological data in anndata](https://mlsanity.com/paper/2604.01949/) (2026)

## Abstract (arXiv)

Training deep learning models on single-cell datasets with hundreds of millions of cells requires loading data from disk, as these datasets exceed available memory. While random sampling provides the data diversity needed for effective training, it is prohibitively slow due to the random access pattern overhead, whereas sequential streaming achieves high throughput but introduces biases that degrade model performance. We present scDataset, a PyTorch data loader that enables efficient training from on-disk data with seamless integration across diverse storage formats. Our approach combines block sampling and batched fetching to achieve quasi-random sampling that balances I/O efficiency with minibatch diversity. On Tahoe-100M, a dataset of 100 million cells, scDataset achieves more than two orders of magnitude speedup compared to true random sampling while working directly with AnnData files. We provide theoretical bounds on minibatch diversity and empirically show that scDataset matches the performance of true random sampling across multiple classification tasks and model architectures.