# Efficient Tabular Data Preprocessing of ML Pipelines

arXiv: https://arxiv.org/abs/2409.14912 · HTML: https://mlsanity.com/paper/2409.14912/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 17 of its 82 references that are in the map.
- First posted on arXiv 23 Sep 2024.

## Builds on (most cited first, 17 of 17)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Neural Collaborative Filtering](https://mlsanity.com/paper/1708.05031/) (2017)
- [BERT4Rec: Sequential Recommendation with Bidirectional Encoder Representations from Transformer](https://mlsanity.com/paper/1904.06690/) (2019)
- [Wide & Deep Learning for Recommender Systems](https://mlsanity.com/paper/1606.07792/) (2016)
- [Yelp Dataset Challenge: Review Rating Prediction](https://mlsanity.com/paper/1605.05362/) (2016)
- [Software-Hardware Co-design for Fast and Scalable Training of Deep Learning Recommendation Models](https://mlsanity.com/paper/2104.05158/) (2021)
- [DeepRecSys: A System for Optimizing End-To-End At-scale Neural Recommendation Inference](https://mlsanity.com/paper/2001.02772/) (2020)
- [tf.data: A Machine Learning Data Processing Framework](https://mlsanity.com/paper/2101.12127/) (2021)
- [Optimizing Deep Learning Recommender Systems' Training On CPU Cluster Architectures](https://mlsanity.com/paper/2005.04680/) (2020)
- [Plumber: Diagnosing and Removing Performance Bottlenecks in Machine Learning Data Pipelines](https://mlsanity.com/paper/2111.04131/) (2021)
- [tf.data service: A Case for Disaggregating ML Input Data Processing](https://mlsanity.com/paper/2210.14826/) (2022)
- [RecD: Deduplication for End-to-End Deep Learning Recommendation Model Training Infrastructure](https://mlsanity.com/paper/2211.05239/) (2022)
- [Farview: Disaggregated Memory with Operator Off-loading for Database Engines](https://mlsanity.com/paper/2106.07102/) (2021)
- [ACCL+: an FPGA-Based Collective Engine for Distributed Applications](https://mlsanity.com/paper/2312.11742/) (2023)
- [Wormhole: A Fast Ordered Index for In-memory Data Management](https://mlsanity.com/paper/1805.02200/) (2018)
- [Disaggregation and the Application](https://mlsanity.com/paper/1910.13056/) (2019)
- [Dagger: Towards Efficient RPCs in Cloud Microservices with Near-Memory Reconfigurable NICs](https://mlsanity.com/paper/2007.08622/) (2020)

## Abstract (arXiv)

Data preprocessing pipelines, which includes data decoding, cleaning, and transforming, are a crucial component of Machine Learning (ML) training. Thy are computationally intensive and often become a major bottleneck, due to the increasing performance gap between the CPUs used for preprocessing and the GPUs used for model training. Recent studies show that a significant number of CPUs across several machines are required to achieve sufficient throughput to saturate the GPUs, leading to increased resource and energy consumption. When the pipeline involves vocabulary generation, the preprocessing performance scales poorly due to significant row-wise synchronization overhead between different CPU cores and servers. To address this limitation, in this paper we present the design of Piper, a hardware accelerator for tabular data preprocessing, prototype it on FPGAs, and demonstrate its potential for training pipelines of commercial recommender systems. Piper achieves 4.7 ~ 71.3× speedup in latency over a 128-core CPU server and outperforms a data-center GPU by 4.8~ 20.3× when using binary input. The impressive performance showcases Piper's potential to increase the efficiency of data preprocessing pipelines and significantly reduce their resource consumption.