# Near-Zero-Overhead Freshness for Recommendation Systems via Inference-Side Model Updates

arXiv: https://arxiv.org/abs/2512.12295 · HTML: https://mlsanity.com/paper/2512.12295/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 26 of its 51 references that are in the map.
- First posted on arXiv 13 Dec 2025.

## Builds on (most cited first, 26 of 26)

- [LoRA: Low-Rank Adaptation of Large Language Models](https://mlsanity.com/paper/2106.09685/) (2021)
- [BERT4Rec: Sequential Recommendation with Bidirectional Encoder Representations from Transformer](https://mlsanity.com/paper/1904.06690/) (2019)
- [Deep Interest Network for Click-Through Rate Prediction](https://mlsanity.com/paper/1706.06978/) (2017)
- [Deep Learning Recommendation Model for Personalization and Recommendation Systems](https://mlsanity.com/paper/1906.00091/) (2019)
- [Embedding-based Retrieval in Facebook Search](https://mlsanity.com/paper/2006.11632/) (2020)
- [FedPara: Low-Rank Hadamard Product for Communication-Efficient Federated Learning](https://mlsanity.com/paper/2108.06098/) (2021)
- [The Architectural Implications of Facebook's DNN-based Personalized Recommendation](https://mlsanity.com/paper/1906.03109/) (2019)
- [Software-Hardware Co-design for Fast and Scalable Training of Deep Learning Recommendation Models](https://mlsanity.com/paper/2104.05158/) (2021)
- [Parameter-Efficient Fine-Tuning with Discrete Fourier Transform](https://mlsanity.com/paper/2405.03003/) (2024)
- [Pixie: A System for Recommending 3+ Billion Items to 200+ Million Users in Real-Time](https://mlsanity.com/paper/1711.07601/) (2017)
- [Understanding Training Efficiency of Deep Learning Recommendation Models at Scale](https://mlsanity.com/paper/2011.05497/) (2020)
- [FBGEMM: Enabling High-Performance Low-Precision Deep Learning Inference](https://mlsanity.com/paper/2101.05615/) (2021)
- [Deep Learning Training in Facebook Data Centers: Design of Scale-up and Scale-out Systems](https://mlsanity.com/paper/2003.09518/) (2020)
- [Monolith: Real Time Recommendation System With Collisionless Embedding Table](https://mlsanity.com/paper/2209.07663/) (2022)
- [HET: Scaling out Huge Embedding Model Training via Cache-enabled Distributed Framework](https://mlsanity.com/paper/2112.07221/) (2021)
- [RecShard: Statistical Feature-Based Memory Optimization for Industry-Scale Neural Recommendation](https://mlsanity.com/paper/2201.10095/) (2022)
- [Dynamically Expandable Graph Convolution for Streaming Recommendation](https://mlsanity.com/paper/2303.11700/) (2023)
- [Addressing Delayed Feedback for Continuous Training with Neural Networks in CTR prediction](https://mlsanity.com/paper/1907.06558/) (2019)
- [Persia: An Open, Hybrid System Scaling Deep Learning-based Recommenders up to 100 Trillion Parameters](https://mlsanity.com/paper/2111.05897/) (2021)
- [Towards Efficient Communication and Secure Federated Recommendation System via Low-rank Training](https://mlsanity.com/paper/2401.03748/) (2024)
- [A GPU-specialized Inference Parameter Server for Large-Scale Deep Recommendation Models](https://mlsanity.com/paper/2210.08804/) (2022)
- [RecD: Deduplication for End-to-End Deep Learning Recommendation Model Training Infrastructure](https://mlsanity.com/paper/2211.05239/) (2022)
- [A Survey on Incremental Update for Neural Recommender Systems](https://mlsanity.com/paper/2303.02851/) (2023)
- [Understanding the Ranking Loss for Recommendation with Sparse User Feedback](https://mlsanity.com/paper/2403.14144/) (2024)
- [Accelerating Distributed ML Training via Selective Synchronization](https://mlsanity.com/paper/2307.07950/) (2023)
- [Supporting Massive DLRM Inference Through Software Defined Memory](https://mlsanity.com/paper/2110.11489/) (2021)

## Built on by (most cited first, 1 of 1)

- [When KV Meets Embeddings: Dynamic GPU Memory Allocation for Accelerating Generative Recommender Serving](https://mlsanity.com/paper/2605.04450/) (2026)

## Abstract (arXiv)

Deep Learning Recommendation Models (DLRMs) underpin personalized services but face a critical freshness-accuracy tradeoff due to massive parameter synchronization overheads. Production DLRMs deploy decoupled training/inference clusters, where synchronizing petabyte-scale embedding tables (EMTs) causes multi-minute staleness, degrading recommendation quality and revenue. We observe that (1) inference nodes exhibit sustained CPU underutilization (peak <= 20%), and (2) EMT gradients possess intrinsic low-rank structure, enabling compact update representation. We present LiveUpdate, a system that eliminates inter-cluster synchronization by colocating Low-Rank Adaptation (LoRA) trainers within inference nodes. LiveUpdate addresses two core challenges: (1) dynamic rank adaptation via singular value monitoring to constrain memory overhead (<2% of EMTs), and (2) NUMA-aware resource scheduling with hardware-enforced QoS to eliminate update inference contention (P99 latency impact <20ms). Evaluations show LiveUpdate reduces update costs by 2x versus delta-update baselines while achieving higher accuracy within 1-hour windows. By transforming idle inference resources into freshness engines, LiveUpdate delivers online model updates while outperforming state-of-the-art delta-update methods by 0.04% to 0.24% in accuracy.