# Preparation Meets Opportunity: Enhancing Data Preprocessing for ML Training With Seneca

arXiv: https://arxiv.org/abs/2511.13724 · HTML: https://mlsanity.com/paper/2511.13724/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 14 of its 55 references that are in the map.
- First posted on arXiv 24 Sep 2025.

## Builds on (most cited first, 14 of 14)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [PyTorch: An Imperative Style, High-Performance Deep Learning Library](https://mlsanity.com/paper/1912.01703/) (2019)
- [Swin Transformer: Hierarchical Vision Transformer using Shifted Windows](https://mlsanity.com/paper/2103.14030/) (2021)
- [Not All Samples Are Created Equal: Deep Learning with Importance Sampling](https://mlsanity.com/paper/1803.00942/) (2018)
- [NVIDIA Tensor Core Programmability, Performance & Precision](https://mlsanity.com/paper/1803.04014/) (2018)
- [tf.data: A Machine Learning Data Processing Framework](https://mlsanity.com/paper/2101.12127/) (2021)
- [Analyzing and Mitigating Data Stalls in DNN Training](https://mlsanity.com/paper/2007.06775/) (2020)
- [Understanding Data Storage and Ingestion for Large-Scale Deep Recommendation Model Training](https://mlsanity.com/paper/2108.09373/) (2021)
- [Plumber: Diagnosing and Removing Performance Bottlenecks in Machine Learning Data Pipelines](https://mlsanity.com/paper/2111.04131/) (2021)
- [tf.data service: A Case for Disaggregating ML Input Data Processing](https://mlsanity.com/paper/2210.14826/) (2022)
- [Clairvoyant Prefetching for Distributed Machine Learning I/O](https://mlsanity.com/paper/2101.08734/) (2021)
- [Where Is My Training Bottleneck? Hidden Trade-Offs in Deep Learning Preprocessing Pipelines](https://mlsanity.com/paper/2202.08679/) (2022)
- [cedar: Optimized and Unified Machine Learning Input Data Pipelines](https://mlsanity.com/paper/2401.08895/) (2024)

## Abstract (arXiv)

Input data preprocessing is a common bottleneck when concurrently training multimedia machine learning (ML) models in modern systems. To alleviate these bottlenecks and reduce the training time for concurrent jobs, we present Seneca, a data loading system that optimizes cache partitioning and data sampling for the data storage and ingestion (DSI) pipeline. The design of Seneca contains two key techniques. First, Seneca uses a performance model for the data pipeline to optimally partition the cache for three different forms of data (encoded, decoded, and augmented). Second, Seneca opportunistically serves cached data over uncached ones during random batch sampling so that concurrent jobs benefit from each other. We implement Seneca by modifying PyTorch and demonstrate its effectiveness by comparing it against several state-of-the-art caching systems for DNN training. Seneca reduces the makespan by 45.23% compared to PyTorch and increases data processing throughput by up to 3.45x compared to the next best dataloader.