mlsanityData to 7 Oct 2026

SA-RSQ: A Versatile Sparse Representation Framework for Multi-modal Recommender Systems

arXiv 2608.22979 ↗First posted 24 Aug 2026cs.AIcited by 0

Not on the map. Why?

Abstract (arXiv)

Deploying high-dimensional multimodal features in industrial recommender systems incurs substantial storage and latency overhead. Hard quantization is compact but introduces boundary distortion, whereas dense soft quantization couples representation quality to the limited storage budget. We propose Sparse Activation-based Residual Soft Quantization (SA-RSQ), which uses Top-K sparse routing and softmax weights to store compact (Index, Probability) tuples. The stored tuples decouple per-item storage from codebook dimensionality; for a fixed selected support, gradients propagate through the routing weights and weighted reconstruction without relying on a straight-through estimator. Experiments on a proprietary food-delivery advertising dataset show favorable reconstruction-performance and CTR trade-offs across storage budgets of 8-48 bytes per item. A preliminary Next-Distribution Prediction study and a one-week online A/B test further demonstrate the practical potential of SA-RSQ, with relative lifts of +2.51% in CTR and +3.66% in CPM.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 27 of its 40 references that are in the map. First posted on arXiv 24 Aug 2026.

Builds on

27 of 40 references.

References · 27

  1. 2016SGDR: Stochastic Gradient Descent with Warm Restartscited by 4,993
  2. 2017Neural Discrete Representation Learningcited by 4,650
  3. 2017Outrageously Large Neural Networks: The Sparsely-Gated Mixture-of-Experts Layercited by 3,413
  4. 2021Switch Transformers: Scaling to Trillion Parameter Models with Simple and Efficient Sparsitycited by 3,014
  5. 2013Estimating or Propagating Gradients Through Stochastic Neurons for Conditional Computationcited by 2,031
  6. 2019Generating Diverse High-Fidelity Images with VQ-VAE-2cited by 1,376
  7. 2022Autoregressive Image Generation using Residual Quantizationcited by 706
  8. 2017Deep Interest Network for Click-Through Rate Predictioncited by 541
  9. 2019Deep Learning Recommendation Model for Personalization and Recommendation Systemscited by 327
  10. 2020Self-supervised Graph Learning for Recommendationcited by 305
  11. 2024Actions Speak Louder than Words: Trillion-Parameter Sequential Transducers for Generative Recommendationscited by 184
  12. 2019Sparse Sequence-to-Sequence Modelscited by 167
15 more
  1. 2025OneRec: Unifying Retrieve and Rank with Generative Recommender and Iterative Preference Alignmentcited by 167
  2. 2017Learning Discrete Representations via Information Maximizing Self-Augmented Trainingcited by 149
  3. 2021Contrastive Learning for Representation Degeneration Problem in Sequential Recommendationcited by 121
  4. 2024Learnable Item Tokenization for Generative Recommendationcited by 106
  5. 2020Controllable Multi-Interest Framework for Recommendationcited by 92
  6. 2024SoftVQ-VAE: Efficient 1-Dimensional Continuous Tokenizercited by 55
  7. 2025OneRec-Think: In-Text Reasoning for Generative Recommendationcited by 47
  8. 2024TokenRec: Learning to Tokenize ID for LLM-based Generative Recommendationcited by 35
  9. 2024Generative Recommender with End-to-End Learnable Item Tokenizationcited by 32
  10. 2021Eider: Empowering Document-level Relation Extraction with Efficient Evidence Extraction and Inference-stage Fusioncited by 28
  11. 2025PLUM: Adapting Pre-trained Language Models for Industrial-scale Generative Recommendationscited by 27
  12. 2024QARM: Quantitative Alignment Multi-Modal Recommendation at Kuaishoucited by 26
  13. 2025Generative Recommendation with Semantic IDs: A Practitioner's Handbookcited by 18
  14. 2025FORGE: Forming Semantic Identifiers for Generative Retrieval in Industrial Datasetscited by 17
  15. 2020Time-based Sequence Model for Personalization and Recommendation Systemscited by 11

Built on by

Not cited yet.