CADC: Encoding User-Item Interactions for Compressing Recommendation Model Training Data
Not on the map. Why?
Abstract (arXiv)
Deep learning recommendation models (DLRMs) are at the heart of the current e-commerce industry. However, the amount of training data used to train these large models is growing exponentially, leading to substantial training hurdles. The training dataset contains two primary types of information: content-based information (features of users and items) and collaborative information (interactions between users and items). One approach to reduce the training dataset is to remove user-item interactions. But that significantly diminishes collaborative information, which is crucial for maintaining accuracy due to its inclusion of interaction histories. This loss profoundly impacts DLRM performance. This paper makes an important observation that if one can capture the user-item interaction history to enrich the user and item embeddings, then the interaction history can be compressed without losing model accuracy. Thus, this work, Collaborative Aware Data Compression (CADC), takes a two-step approach to training dataset compression. In the first step, we use matrix factorization of the user-item interaction matrix to create a novel embedding representation for both the users and items. Once the user and item embeddings are enriched by the interaction history information the approach then applies uniform random sampling of the training dataset to drastically reduce the training dataset size while minimizing model accuracy drop. The source code of CADC is available at hrefhttps://anonymous.4open.science/r/DSS-RM-8C1D/README.mdhttps://anonymous.4open.science/r/DSS-RM-8C1D/README.md.
Key facts. Cited by 1 paper in the map (data through 14 Oct 2026). Builds on 27 of its 43 references that are in the map. First posted on arXiv 11 Jul 2024.
Builds on
27 of 43 references.
References · 27
- 2022GPTQ: Accurate Post-Training Quantization for Generative Pre-trained Transformerscited by 1,919
- 2018Graph Convolutional Neural Networks for Web-Scale Recommender Systemscited by 1,138
- 2016Wide & Deep Learning for Recommender Systemscited by 747
- 2018An Empirical Study of Example Forgetting during Deep Neural Network Learningcited by 678
- 2019Selection via Proxy: Efficient Data Selection for Deep Learningcited by 339
- 2018Deep Interest Evolution Network for Click-Through Rate Predictioncited by 338
- 2019Deep Learning Recommendation Model for Personalization and Recommendation Systemscited by 327
- 2021Sustainable AI: Environmental Implications, Challenges and Opportunitiescited by 242
- 2021Dataset Distillation with Infinitely Wide Convolutional Networkscited by 218
- 2021PTQ4ViT: Post-training quantization for vision transformers with twin uniform quantizationcited by 179
- 2022A Fast Post-Training Pruning Framework for Transformerscited by 162
- 2021Graph Condensation for Graph Neural Networkscited by 150
15 more
- 2023LoRA-FA: Efficient and Effective Low Rank Representation Fine-tuningcited by 137
- 2017On Sampling Strategies for Neural Network-based Collaborative Filteringcited by 71
- 2020Distributed Hierarchical GPU Parameter Server for Massive Scale Deep Learning Ads Systemscited by 60
- 2022Adaptive Second Order Coresets for Data-efficient Machine Learningcited by 60
- 2020DeepRecSys: A System for Optimizing End-To-End At-scale Neural Recommendation Inferencecited by 54
- 2019Ordered SGD: A New Stochastic Optimization Framework for Empirical Risk Minimizationcited by 51
- 2020Understanding Training Efficiency of Deep Learning Recommendation Models at Scalecited by 43
- 2020Deep Learning Training in Facebook Data Centers: Design of Scale-up and Scale-out Systemscited by 38
- 2020A Model of Two Tales: Dual Transfer Learning Framework for Improved Long-tail Item Recommendationcited by 38
- 2021ECLARE: Extreme Classification with Label Graph Correlationscited by 38
- 2020Towards Automated Neural Interaction Discovery for Click-Through Rate Predictioncited by 37
- 2020Understanding Capacity-Driven Scale-Out Neural Recommendation Inferencecited by 29
- 2022RecShard: Statistical Feature-Based Memory Optimization for Industry-Scale Neural Recommendationcited by 28
- 2024LaMDA: Large Model Fine-Tuning via Spectrally Decomposed Low-Dimensional Adaptationcited by 16
- 2024PEANO-ViT: Power-Efficient Approximations of Non-Linearities in Vision Transformerscited by 13
Built on by
Citing papers per year: 2024: 1.
1 paper.