mlsanityData to 7 Oct 2026

Adaptive Low-Precision Training for Embeddings in Click-Through Rate Prediction

arXiv 2212.05735 ↗First posted 12 Dec 2022cs.LG cs.IRcited by 11

Not on the map. Why?

Abstract (arXiv)

Embedding tables are usually huge in click-through rate (CTR) prediction models. To train and deploy the CTR models efficiently and economically, it is necessary to compress their embedding tables at the training stage. To this end, we formulate a novel quantization training paradigm to compress the embeddings from the training stage, termed low-precision training (LPT). Also, we provide theoretical analysis on its convergence. The results show that stochastic weight quantization has a faster convergence rate and a smaller convergence error than deterministic weight quantization in LPT. Further, to reduce the accuracy degradation, we propose adaptive low-precision training (ALPT) that learns the step size (i.e., the quantization resolution) through gradient descent. Experiments on two real-world datasets confirm our analysis and show that ALPT can significantly improve the prediction accuracy, especially at extremely low bit widths. For the first time in CTR models, we successfully train 8-bit embeddings without sacrificing prediction accuracy. The code of ALPT is publicly available.

Key facts. Cited by 11 papers in the map (data through 7 Oct 2026). Builds on 30 of its 39 references that are in the map. First posted on arXiv 12 Dec 2022.

Builds on

30 of 39 references.

References · 30

  1. 2014Adam: A Method for Stochastic Optimizationcited by 36,960
  2. 2018DARTS: Differentiable Architecture Searchcited by 1,782
  3. 2016XNOR-Net: ImageNet Classification Using Binary Convolutional Neural Networkscited by 829
  4. 2016Wide & Deep Learning for Recommender Systemscited by 745
  5. 2017DeepFM: A Factorization-Machine based Neural Network for CTR Predictioncited by 673
  6. 2019Learned Step Size Quantizationcited by 604
  7. 2020Up or Down? Adaptive Rounding for Post-Training Quantizationcited by 600
  8. 2015BinaryConnect: Training Deep Neural Networks with binary weights during propagationscited by 571
  9. 2017Deep Interest Network for Click-Through Rate Predictioncited by 541
  10. 2018PACT: Parameterized Clipping Activation for Quantized Neural Networkscited by 456
  11. 2017Deep & Cross Network for Ad Click Predictionscited by 392
  12. 2017Training Quantized Nets: A Deeper Understandingcited by 74
18 more
  1. 2019Compositional Embeddings Using Complementary Partitions for Memory-Efficient Recommendation Systemscited by 68
  2. 2021TT-Rec: Tensor Train Compression for Deep Learning Recommendation Modelscited by 58
  3. 2016Loss-aware Binarization of Deep Networkscited by 56
  4. 2020Open Benchmarking for Click-Through Rate Predictioncited by 53
  5. 2015Neural Networks with Few Multiplicationscited by 46
  6. 2021Learnable Embedding Sizes for Recommender Systemscited by 42
  7. 2019Neural Input Search for Large Scale Recommendation Modelscited by 39
  8. 2020An Embedding Learning Framework for Numerical Features in CTR Predictioncited by 37
  9. 2020Model Size Reduction Using Frequency Based Double Hashing for Recommender Systemscited by 33
  10. 2018Loss-aware Weight Quantization of Deep Networkscited by 31
  11. 2020Detecting Beneficial Feature Interactions for Recommender Systemscited by 22
  12. 2019Post-Training 4-bit Quantization on Embedding Tablescited by 21
  13. 2022On-Device Next-Item Recommendation with Self-Supervised Knowledge Distillationcited by 21
  14. 2021Learning Elastic Embeddings for Customizing On-Device Recommenderscited by 19
  15. 2020Mixed-Precision Embedding Using a Cachecited by 18
  16. 2020Memory-efficient Embedding for Recommendationscited by 15
  17. 2022OptEmbed: Learning Optimal Embedding Table for Click-through Rate Predictioncited by 14
  18. 2021ScaleFreeCTR: MixCache-based Distributed Training System for CTR Models with Huge Embedding Tablecited by 12

Built on by

Cited by year

Citing papers per year: 2023: 1; 2024: 2; 2025: 8.

11 papers.

  1. 2025Beyond Zero Initialization: Investigating the Impact of Non-Zero Initialization on LoRA Fine-Tuning Dynamicscited by 16
  2. 2023CAFE: Towards Compact, Adaptive, and Fast Embedding for Large-scale Recommendation Modelscited by 7
  3. 2025Semantic Retrieval Augmented Contrastive Learning for Sequential Recommendationcited by 6
  4. 2025The Panaceas for Improving Low-Rank Decomposition in Communication-Efficient Federated Learningcited by 5
  5. 2025Beyond Higher Rank: Token-wise Input-Output Projections for Efficient Low-Rank Adaptationcited by 5
  6. 2024A Thorough Performance Benchmarking on Lightweight Embedding-based Recommender Systemscited by 4
  7. 2025The Future is Sparse: Embedding Compression for Scalable Retrieval in Recommender Systemscited by 4
  8. 2025FedBiF: Communication-Efficient Federated Learning via Bits Freezingcited by 1
  9. 2024ProactivePIM: Accelerating Weight-Sharing Embedding Layer with PIM for Scalable Recommendation Systemcited by 0
  10. 2025ShuffleGate: Scalable Feature Optimization for Recommender Systems via Batch-wise Sensitivity Learningcited by 0
  11. 2025DimGrow: Memory-Efficient Field-level Embedding Dimension Searchcited by 0