mlsanityData to 7 Oct 2026

Near-Zero-Overhead Freshness for Recommendation Systems via Inference-Side Model Updates

arXiv 2512.12295 ↗First posted 13 Dec 2025cs.DC cs.LGcited by 1

Not on the map. Why?

Abstract (arXiv)

Deep Learning Recommendation Models (DLRMs) underpin personalized services but face a critical freshness-accuracy tradeoff due to massive parameter synchronization overheads. Production DLRMs deploy decoupled training/inference clusters, where synchronizing petabyte-scale embedding tables (EMTs) causes multi-minute staleness, degrading recommendation quality and revenue. We observe that (1) inference nodes exhibit sustained CPU underutilization (peak <= 20%), and (2) EMT gradients possess intrinsic low-rank structure, enabling compact update representation. We present LiveUpdate, a system that eliminates inter-cluster synchronization by colocating Low-Rank Adaptation (LoRA) trainers within inference nodes. LiveUpdate addresses two core challenges: (1) dynamic rank adaptation via singular value monitoring to constrain memory overhead (<2% of EMTs), and (2) NUMA-aware resource scheduling with hardware-enforced QoS to eliminate update inference contention (P99 latency impact <20ms). Evaluations show LiveUpdate reduces update costs by 2x versus delta-update baselines while achieving higher accuracy within 1-hour windows. By transforming idle inference resources into freshness engines, LiveUpdate delivers online model updates while outperforming state-of-the-art delta-update methods by 0.04% to 0.24% in accuracy.

Key facts. Cited by 1 paper in the map (data through 7 Oct 2026). Builds on 26 of its 51 references that are in the map. First posted on arXiv 13 Dec 2025.

Builds on

26 of 51 references.

References · 26

  1. 2021LoRA: Low-Rank Adaptation of Large Language Modelscited by 14,084
  2. 2019BERT4Rec: Sequential Recommendation with Bidirectional Encoder Representations from Transformercited by 952
  3. 2017Deep Interest Network for Click-Through Rate Predictioncited by 541
  4. 2019Deep Learning Recommendation Model for Personalization and Recommendation Systemscited by 327
  5. 2020Embedding-based Retrieval in Facebook Searchcited by 155
  6. 2021FedPara: Low-Rank Hadamard Product for Communication-Efficient Federated Learningcited by 140
  7. 2019The Architectural Implications of Facebook's DNN-based Personalized Recommendationcited by 83
  8. 2021Software-Hardware Co-design for Fast and Scalable Training of Deep Learning Recommendation Modelscited by 78
  9. 2024Parameter-Efficient Fine-Tuning with Discrete Fourier Transformcited by 67
  10. 2017Pixie: A System for Recommending 3+ Billion Items to 200+ Million Users in Real-Timecited by 46
  11. 2020Understanding Training Efficiency of Deep Learning Recommendation Models at Scalecited by 43
  12. 2021FBGEMM: Enabling High-Performance Low-Precision Deep Learning Inferencecited by 40
14 more
  1. 2020Deep Learning Training in Facebook Data Centers: Design of Scale-up and Scale-out Systemscited by 37
  2. 2022Monolith: Real Time Recommendation System With Collisionless Embedding Tablecited by 30
  3. 2021HET: Scaling out Huge Embedding Model Training via Cache-enabled Distributed Frameworkcited by 28
  4. 2022RecShard: Statistical Feature-Based Memory Optimization for Industry-Scale Neural Recommendationcited by 28
  5. 2023Dynamically Expandable Graph Convolution for Streaming Recommendationcited by 26
  6. 2019Addressing Delayed Feedback for Continuous Training with Neural Networks in CTR predictioncited by 25
  7. 2021Persia: An Open, Hybrid System Scaling Deep Learning-based Recommenders up to 100 Trillion Parameterscited by 17
  8. 2024Towards Efficient Communication and Secure Federated Recommendation System via Low-rank Trainingcited by 14
  9. 2022A GPU-specialized Inference Parameter Server for Large-Scale Deep Recommendation Modelscited by 9
  10. 2022RecD: Deduplication for End-to-End Deep Learning Recommendation Model Training Infrastructurecited by 9
  11. 2023A Survey on Incremental Update for Neural Recommender Systemscited by 8
  12. 2024Understanding the Ranking Loss for Recommendation with Sparse User Feedbackcited by 8
  13. 2023Accelerating Distributed ML Training via Selective Synchronizationcited by 7
  14. 2021Supporting Massive DLRM Inference Through Software Defined Memorycited by 5

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2026: 1.

1 paper.

  1. 2026When KV Meets Embeddings: Dynamic GPU Memory Allocation for Accelerating Generative Recommender Servingcited by 0