# CADC: Encoding User-Item Interactions for Compressing Recommendation Model Training Data

arXiv: https://arxiv.org/abs/2407.08108 · HTML: https://mlsanity.com/paper/2407.08108/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 27 of its 43 references that are in the map.
- First posted on arXiv 11 Jul 2024.

## Builds on (most cited first, 27 of 27)

- [GPTQ: Accurate Post-Training Quantization for Generative Pre-trained Transformers](https://mlsanity.com/paper/2210.17323/) (2022)
- [Graph Convolutional Neural Networks for Web-Scale Recommender Systems](https://mlsanity.com/paper/1806.01973/) (2018)
- [Wide & Deep Learning for Recommender Systems](https://mlsanity.com/paper/1606.07792/) (2016)
- [An Empirical Study of Example Forgetting during Deep Neural Network Learning](https://mlsanity.com/paper/1812.05159/) (2018)
- [Selection via Proxy: Efficient Data Selection for Deep Learning](https://mlsanity.com/paper/1906.11829/) (2019)
- [Deep Interest Evolution Network for Click-Through Rate Prediction](https://mlsanity.com/paper/1809.03672/) (2018)
- [Deep Learning Recommendation Model for Personalization and Recommendation Systems](https://mlsanity.com/paper/1906.00091/) (2019)
- [Sustainable AI: Environmental Implications, Challenges and Opportunities](https://mlsanity.com/paper/2111.00364/) (2021)
- [Dataset Distillation with Infinitely Wide Convolutional Networks](https://mlsanity.com/paper/2107.13034/) (2021)
- [PTQ4ViT: Post-training quantization for vision transformers with twin uniform quantization](https://mlsanity.com/paper/2111.12293/) (2021)
- [A Fast Post-Training Pruning Framework for Transformers](https://mlsanity.com/paper/2204.09656/) (2022)
- [Graph Condensation for Graph Neural Networks](https://mlsanity.com/paper/2110.07580/) (2021)
- [LoRA-FA: Efficient and Effective Low Rank Representation Fine-tuning](https://mlsanity.com/paper/2308.03303/) (2023)
- [On Sampling Strategies for Neural Network-based Collaborative Filtering](https://mlsanity.com/paper/1706.07881/) (2017)
- [Distributed Hierarchical GPU Parameter Server for Massive Scale Deep Learning Ads Systems](https://mlsanity.com/paper/2003.05622/) (2020)
- [Adaptive Second Order Coresets for Data-efficient Machine Learning](https://mlsanity.com/paper/2207.13887/) (2022)
- [DeepRecSys: A System for Optimizing End-To-End At-scale Neural Recommendation Inference](https://mlsanity.com/paper/2001.02772/) (2020)
- [Ordered SGD: A New Stochastic Optimization Framework for Empirical Risk Minimization](https://mlsanity.com/paper/1907.04371/) (2019)
- [Understanding Training Efficiency of Deep Learning Recommendation Models at Scale](https://mlsanity.com/paper/2011.05497/) (2020)
- [Deep Learning Training in Facebook Data Centers: Design of Scale-up and Scale-out Systems](https://mlsanity.com/paper/2003.09518/) (2020)
- [A Model of Two Tales: Dual Transfer Learning Framework for Improved Long-tail Item Recommendation](https://mlsanity.com/paper/2010.15982/) (2020)
- [ECLARE: Extreme Classification with Label Graph Correlations](https://mlsanity.com/paper/2108.00261/) (2021)
- [Towards Automated Neural Interaction Discovery for Click-Through Rate Prediction](https://mlsanity.com/paper/2007.06434/) (2020)
- [Understanding Capacity-Driven Scale-Out Neural Recommendation Inference](https://mlsanity.com/paper/2011.02084/) (2020)
- [RecShard: Statistical Feature-Based Memory Optimization for Industry-Scale Neural Recommendation](https://mlsanity.com/paper/2201.10095/) (2022)
- [LaMDA: Large Model Fine-Tuning via Spectrally Decomposed Low-Dimensional Adaptation](https://mlsanity.com/paper/2406.12832/) (2024)
- [PEANO-ViT: Power-Efficient Approximations of Non-Linearities in Vision Transformers](https://mlsanity.com/paper/2406.14854/) (2024)

## Built on by (most cited first, 1 of 1)

- [CHOSEN: Compilation to Hardware Optimization Stack for Efficient Vision Transformer Inference](https://mlsanity.com/paper/2407.12736/) (2024)

## Abstract (arXiv)

Deep learning recommendation models (DLRMs) are at the heart of the current e-commerce industry. However, the amount of training data used to train these large models is growing exponentially, leading to substantial training hurdles. The training dataset contains two primary types of information: content-based information (features of users and items) and collaborative information (interactions between users and items). One approach to reduce the training dataset is to remove user-item interactions. But that significantly diminishes collaborative information, which is crucial for maintaining accuracy due to its inclusion of interaction histories. This loss profoundly impacts DLRM performance. This paper makes an important observation that if one can capture the user-item interaction history to enrich the user and item embeddings, then the interaction history can be compressed without losing model accuracy. Thus, this work, Collaborative Aware Data Compression (CADC), takes a two-step approach to training dataset compression. In the first step, we use matrix factorization of the user-item interaction matrix to create a novel embedding representation for both the users and items. Once the user and item embeddings are enriched by the interaction history information the approach then applies uniform random sampling of the training dataset to drastically reduce the training dataset size while minimizing model accuracy drop. The source code of CADC is available at hrefhttps://anonymous.4open.science/r/DSS-RM-8C1D/README.mdhttps://anonymous.4open.science/r/DSS-RM-8C1D/README.md.