mlsanityData to 7 Oct 2026

The Efficiency vs. Accuracy Trade-off: Optimizing RAG-Enhanced LLM Recommender Systems Using Multi-Head Early Exit

arXiv 2501.02173 ↗First posted 4 Jan 2025cs.IR cs.LGcited by 10

On the map: Efficient LLM inference and serving → Prompt and context compression

Abstract (arXiv)

The deployment of Large Language Models (LLMs) in recommender systems for predicting Click-Through Rates (CTR) necessitates a delicate balance between computational efficiency and predictive accuracy. This paper presents an optimization framework that combines Retrieval-Augmented Generation (RAG) with an innovative multi-head early exit architecture to concurrently enhance both aspects. By integrating Graph Convolutional Networks (GCNs) as efficient retrieval mechanisms, we are able to significantly reduce data retrieval times while maintaining high model performance. The early exit strategy employed allows for dynamic termination of model inference, utilizing real-time predictive confidence assessments across multiple heads. This not only quickens the responsiveness of LLMs but also upholds or improves their accuracy, making it ideal for real-time application scenarios. Our experiments demonstrate how this architecture effectively decreases computation time without sacrificing the accuracy needed for reliable recommendation delivery, establishing a new standard for efficient, real-time LLM deployment in commercial systems.

Key facts. Cited by 10 papers in the map (data through 7 Oct 2026). Placed in Prompt and context compression, one of 12 directions in Efficient LLM inference and serving. Builds on 44 of its 61 references that are in the map. First posted on arXiv 4 Jan 2025.

Builds on

44 of 61 references.

References · 44

  1. 2016Semi-Supervised Classification with Graph Convolutional Networkscited by 11,282
  2. 2023SparseGPT: Massive Language Models Can Be Accurately Pruned in One-Shotcited by 1,090
  3. 2020Simple and Deep Graph Convolutional Networkscited by 857
  4. 2024Medusa: Simple LLM Inference Acceleration Framework with Multiple Decoding Headscited by 821
  5. 2023A Simple and Effective Pruning Approach for Large Language Modelscited by 776
  6. 2015Session-based Recommendations with Recurrent Neural Networkscited by 685
  7. 2017DeepFM: A Factorization-Machine based Neural Network for CTR Predictioncited by 673
  8. 2023LLM-Pruner: On the Structural Pruning of Large Language Modelscited by 643
  9. 2017Deep Interest Network for Click-Through Rate Predictioncited by 541
  10. 2023Sheared LLaMA: Accelerating Language Model Pre-training via Structured Pruningcited by 408
  11. 2023A Survey on Large Language Models for Recommendationcited by 378
  12. 2023TALLRec: An Effective and Efficient Tuning Framework to Align Large Language Model with Recommendationcited by 336
32 more
  1. 2020DCN V2: Improved Deep & Cross Network and Practical Lessons for Web-scale Learning to Rank Systemscited by 326
  2. 2018AutoInt: Automatic Feature Interaction Learning via Self-Attentive Neural Networkscited by 319
  3. 2024LayerSkip: Enabling Early Exit Inference and Self-Speculative Decodingcited by 302
  4. 2022Confident Adaptive Language Modelingcited by 297
  5. 2020BERT Loses Patience: Fast and Robust Inference with Early Exitcited by 267
  6. 2024Unlocking Efficiency in Large Language Model Inference: A Comprehensive Survey of Speculative Decodingcited by 266
  7. 2020Training with Quantization Noise for Extreme Model Compressioncited by 145
  8. 2024Kangaroo: A Powerful Video-Language Model Supporting Long-context Video Inputcited by 132
  9. 2022Less is More: Task-aware Layer-wise Distillation for Language Model Compressioncited by 91
  10. 2023Fast and Robust Early-Exiting Framework for Autoregressive Language Models with Synchronized Parallel Decodingcited by 80
  11. 2022Compression of Generative Pre-trained Language Models via Quantizationcited by 79
  12. 2023Leveraging Large Language Models for Sequential Recommendationcited by 78
  13. 2023ReLLa: Retrieval-enhanced Large Language Models for Lifelong Sequential Behavior Comprehension in Recommendationcited by 76
  14. 2024Wukong: Towards a Scaling Law for Large-Scale Recommendationcited by 72
  15. 2023EE-LLM: Large-Scale Training and Inference of Early-Exit Large Language Models with 3D Parallelismcited by 69
  16. 2023Collaborative Large Language Model for Recommender Systemscited by 63
  17. 2024Kangaroo: Lossless Self-Speculative Decoding via Double Early Exitingcited by 58
  18. 2023Zero-Shot Next-Item Recommendation using Large Pretrained Language Modelscited by 53
  19. 2024Let Me Do It For You: Towards LLM Empowered Recommendation via Tool Learningcited by 51
  20. 2024Large Language Models meet Collaborative Filtering: An Efficient All-round LLM-based Recommender Systemcited by 50
  21. 2024A Comprehensive Evaluation of Quantization Strategies for Large Language Modelscited by 47
  22. 2024Harnessing Large Language Models for Text-Rich Sequential Recommendationcited by 38
  23. 2021Towards Efficient Post-training Quantization of Pre-trained Language Modelscited by 37
  24. 2023KuaiSim: A Comprehensive Simulator for Recommender Systemscited by 30
  25. 2021Accelerating BERT Inference for Sequence Labeling via Early-Exitcited by 29
  26. 2023EulerNet: Adaptive Feature Interaction Learning via Euler's Formula for CTR Predictioncited by 26
  27. 2023PromptMix: A Class Boundary Augmentation Method for Large Language Model Distillationcited by 25
  28. 2023Towards Deeper, Lighter and Interpretable Cross Network for CTR Predictioncited by 20
  29. 2024Ad Auctions for LLMs via Retrieval Augmented Generationcited by 17
  30. 2023Generative Job Recommendations with Large Language Modelcited by 13
  31. 2024TinyLLM: A Framework for Training and Deploying Language Models at the Edge Computerscited by 9
  32. 2023BERT4CTR: An Efficient Framework to Combine Pre-trained Language Model with Non-textual Features for CTR Predictioncited by 8

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 3; 2026: 7.

10 papers.

  1. 2025A Survey of Large Language Model Empowered Agents for Recommendation and Search: Towards Next-Generation Information Retrievalcited by 31
  2. 2026GR2: Generative Reasoning Re-rankercited by 11
  3. 2025Graph Foundation Models for Recommendation: A Comprehensive Surveycited by 3
  4. 2026Autonomous Information Seeking: A Roadmap for Agentic Recommender Systemscited by 3
  5. 2025DND: Boosting Large Language Models with Dynamic Nested Depthcited by 1
  6. 2026PRISM: Purified Representation and Integrated Semantic Modeling for Generative Sequential Recommendationcited by 0
  7. 2026Mitigating Hallucinations in Large Language Models Via Decoder Layer Skippingcited by 0
  8. 2026GR2 Technical Reportcited by 0
  9. 2026Diffusion-GR2: Diffusion Generative Reasoning Re-rankercited by 0
  10. 2026Mapping the RAG Landscape: A Four Axis Taxonomy of Efficiency, Defense, Interactivity, and Reasoningcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025A Comprehensive Review on Harnessing Large Language Models to Overcome Recommender System Challenges
  2. 2025Balancing Fine-tuning and RAG: A Hybrid Strategy for Dynamic LLM Recommendation Updates
  3. 2025Knowledge Graph Retrieval-Augmented Generation for LLM-based Recommendation
  4. 2025Selective LLM-Guided Regularization for Enhancing Recommendation Models
  5. 2025Rethinking LLM-Based Recommendations: A Personalized Query-Driven Parallel Integration
  6. 2025WebRec: Enhancing LLM-based Recommendations with Attention-guided RAG from Web
  7. 2025Language Ranker: A Lightweight Ranking framework for LLM Decoding
  8. 2024Rethinking Large Language Model Architectures for Sequential Recommendations