# tf.data service: A Case for Disaggregating ML Input Data Processing arXiv: https://arxiv.org/abs/2210.14826 ยท HTML: https://mlsanity.com/paper/2210.14826/ ## Key facts - Cited by 16 papers in the map (data through 14 Oct 2026). - Builds on 15 of its 62 references that are in the map. - First posted on arXiv 26 Oct 2022. ## Builds on (most cited first, 15 of 15) - [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015) - [PyTorch: An Imperative Style, High-Performance Deep Learning Library](https://mlsanity.com/paper/1912.01703/) (2019) - [Microsoft COCO: Common Objects in Context](https://mlsanity.com/paper/1405.0312/) (2014) - [RandAugment: Practical automated data augmentation with a reduced search space](https://mlsanity.com/paper/1909.13719/) (2019) - [In-Datacenter Performance Analysis of a Tensor Processing Unit](https://mlsanity.com/paper/1704.04760/) (2017) - [A Survey on Distributed Machine Learning](https://mlsanity.com/paper/1912.09789/) (2019) - [tf.data: A Machine Learning Data Processing Framework](https://mlsanity.com/paper/2101.12127/) (2021) - [How Much Data Are Augmentations Worth? An Investigation into Scaling Laws, Invariance, and Implicit Regularization](https://mlsanity.com/paper/2210.06441/) (2022) - [Analyzing and Mitigating Data Stalls in DNN Training](https://mlsanity.com/paper/2007.06775/) (2020) - [Understanding Data Storage and Ingestion for Large-Scale Deep Recommendation Model Training](https://mlsanity.com/paper/2108.09373/) (2021) - [Neural Data Server: A Large-Scale Search Engine for Transfer Learning Data](https://mlsanity.com/paper/2001.02799/) (2020) - [Faster Neural Network Training with Data Echoing](https://mlsanity.com/paper/1907.05550/) (2019) - [Plumber: Diagnosing and Removing Performance Bottlenecks in Machine Learning Data Pipelines](https://mlsanity.com/paper/2111.04131/) (2021) - [Scalable Neural Data Server: A Data Recommender for Transfer Learning](https://mlsanity.com/paper/2206.09386/) (2022) - [Pay One, Get Hundreds for Free: Reducing Cloud Costs through Shared Query Execution](https://mlsanity.com/paper/1809.00159/) (2018) ## Built on by (most cited first, 16 of 16) - [Characterization of Large Language Model Development in the Datacenter](https://mlsanity.com/paper/2403.07648/) (2024) - [cedar: Optimized and Unified Machine Learning Input Data Pipelines](https://mlsanity.com/paper/2401.08895/) (2024) - [TensorSocket: Shared Data Loading for Deep Learning Training](https://mlsanity.com/paper/2409.18749/) (2024) - [MegaScale-Data: Scaling Dataloader for Multisource Large Foundation Model Training](https://mlsanity.com/paper/2504.09844/) (2025) - [I/O in Machine Learning Applications on HPC Systems: A 360-degree Survey](https://mlsanity.com/paper/2404.10386/) (2024) - [Hydraulis: Balancing Large Transformer Model Training via Co-designing Parallel Strategies and Data Assignment](https://mlsanity.com/paper/2412.07894/) (2024) - [The Streaming Batch Model for Efficient and Fault-Tolerant Heterogeneous Execution](https://mlsanity.com/paper/2501.12407/) (2025) - [PreSto: An In-Storage Data Preprocessing System for Training Recommendation Models](https://mlsanity.com/paper/2406.14571/) (2024) - [Position: Neglecting the Sustainability of AI is Fuelling a Global AI Arms Race](https://mlsanity.com/paper/2502.20016/) (2025) - [Bullion: A Column Store for Machine Learning](https://mlsanity.com/paper/2404.08901/) (2024) - [Efficient Tabular Data Preprocessing of ML Pipelines](https://mlsanity.com/paper/2409.14912/) (2024) - [Accelerating Recommender Model ETL with a Streaming FPGA-GPU Dataflow](https://mlsanity.com/paper/2501.12032/) (2025) - [Preparation Meets Opportunity: Enhancing Data Preprocessing for ML Training With Seneca](https://mlsanity.com/paper/2511.13724/) (2025) - [BatchWeave: A Consistent Object-Store-Native Data Plane for Large Foundation Model Training](https://mlsanity.com/paper/2605.09994/) (2026) - [Minibatch persistency, eight years later: what batch reuse costs in steps and joules, and what it saves in data](https://mlsanity.com/paper/2609.13922/) (2026) - [Zephon: Elastic Determinism for Online, Stateful Foundation Model Data Loading Pipelines](https://mlsanity.com/paper/2610.03087/) (2026) ## Abstract (arXiv) Machine learning (ML) computations commonly execute on expensive specialized hardware, such as GPUs and TPUs, which provide high FLOPs and performance-per-watt. For cost efficiency, it is essential to keep these accelerators highly utilized. This requires preprocessing input data at the rate at which the accelerators can ingest and perform ML computations on the data. To avoid data stalls, the host CPU and RAM required for input data processing per accelerator core used for ML computations varies across jobs. Hence, the traditional approach of processing input data on ML accelerator hosts with a fixed hardware ratio leads to either under-utilizing the accelerators or the host CPU and RAM. In this paper, we address these concerns by building a disaggregated ML data processing system. We present tf.data service, an open-source disaggregated input data processing service built on top of tf.data in TensorFlow. We show that disaggregating data preprocessing has three key advantages for large-scale ML training jobs. First, the service can horizontally scale-out to right-size CPU/RAM host resources for data processing in each job, saving 32x training time and 26x cost, on average. Second, the service can share ephemeral preprocessed data results across jobs, to optimize CPU usage and reduce redundant computations. Finally, the service supports coordinated reads, a technique that avoids stragglers due to different input sizes in distributed training, reducing training time by 2.2x, on average. Our design is inspired by lessons learned from deploying tf.data service in production, including relaxing data visitation guarantees without impacting model accuracy.