mlsanityData to 7 Oct 2026

Towards Efficient Reasoning in LLM-Based Recommender Systems via Model Merging

arXiv 2608.10447 ↗First posted 11 Aug 2026cs.IR cs.AIcited by 0

On the map: LLM reasoning → LLM reasoning for recommendation

Abstract (arXiv)

Large language model-based recommender systems are increasingly adopting slow-thinking models that generate step-by-step reasoning before making predictions, often achieving higher accuracy than fast-thinking models that predict directly. However, their reasoning traces are often unnecessarily verbose, increasing inference costs without commensurate accuracy gains. Existing training-based approaches to reasoning compression often incur substantial adaptation costs, while inference-time methods are brittle and difficult to scale. These limitations motivate model merging as a promising training-free direction for transferring specialised behaviours between models in a shared parameter space. In particular, merging a slow-thinking model with a fast-thinking counterpart provides a natural mechanism for balancing recommendation accuracy and reasoning conciseness. To this end, we propose, to our knowledge, the first model merging framework for reasoning compression in recommender systems. Unlike conventional merging methods that apply uniform merge coefficients across model components, our method performs fine-grained merging at the level of individual attention heads, capturing heterogeneous patterns in recommendation reasoning. Each attention head is assigned a distinct merge coefficient according to its contribution to critical reasoning evidence and its sensitivity to parameter change, enabling selective injection of the concise behaviour of the fast-thinking model into the slow-thinking model and reducing reasoning verbosity without compromising recommendation quality. Experiments on three benchmark datasets show that our method reduces reasoning length by up to 24.3% while outperforming competitive model merging baselines in maintaining recommendation accuracy. The code is available at https://github.com/linhledieu/REAM.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Placed in LLM reasoning for recommendation, one of 17 directions in LLM reasoning. Builds on 45 of its 55 references that are in the map. First posted on arXiv 11 Aug 2026.

Builds on

45 of 55 references.

Closest ancestors · 6

Same direction, newest first

  1. 2026Reinforced Preference Optimization for Reasoning-Augmented Recommendations21 May
  2. 2025SCoTER: Structured Chain-of-Thought Transfer for Enhanced Recommendation24 Nov
  3. 2025Think before Recommendation: Autonomous Reasoning-enhanced Recommender27 Oct
  4. 2025Reinforced Latent Reasoning for LLM-based Recommendation25 May
  5. 2025Reason4Rec: Deliberative User Preference Alignment of Large Language Models for Recommendation4 Feb
  6. 2024Large Language Models Enhanced Collaborative Filtering26 Mar

Other references · 39

  1. 2021LoRA: Low-Rank Adaptation of Large Language Modelscited by 14,084
  2. 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,302
  3. 2022Model soups: averaging weights of multiple fine-tuned models improves accuracy without increasing inference timecited by 1,366
  4. 2020Transformer Feed-Forward Layers Are Key-Value Memoriescited by 1,307
  5. 2022Editing Models with Task Arithmeticcited by 1,076
  6. 2019Analyzing Multi-Head Self-Attention: Specialized Heads Do the Heavy Lifting, the Rest Can Be Prunedcited by 844
  7. 2023TIES-Merging: Resolving Interference When Merging Modelscited by 795
  8. 2023Language Models are Super Mario: Absorbing Abilities from Homologous Models as a Free Lunchcited by 679
  9. 2021Merging Models with Fisher-Weighted Averagingcited by 613
  10. 2025Stop Overthinking: A Survey on Efficient Reasoning for Large Language Modelscited by 456
  11. 2014New insights and perspectives on the natural gradient methodcited by 435
  12. 2022Recommendation as Language Processing (RLP): A Unified Pretrain, Personalized Prompt & Predict Paradigm (P5)cited by 396
27 more
  1. 2025L1: Controlling How Long A Reasoning Model Thinks With Reinforcement Learningcited by 381
  2. 2023A Survey on Large Language Models for Recommendationcited by 378
  3. 2025Chain of Draft: Thinking Faster by Writing Lesscited by 290
  4. 2024Model Merging in LLMs, MLLMs, and Beyond: Methods, Theories, Applications and Opportunitiescited by 247
  5. 2025TokenSkip: Controllable Chain-of-Thought Compression in LLMscited by 246
  6. 2024Token-Budget-Aware LLM Reasoningcited by 242
  7. 2024Retrieval Head Mechanistically Explains Long-Context Factualitycited by 219
  8. 2019Limitations of the Empirical Fisher Approximation for Natural Gradient Descentcited by 213
  9. 2025ThinkPrune: Pruning Long Chain-of-Thought of LLMs via Reinforcement Learningcited by 187
  10. 2023Recommendation as Instruction Following: A Large Language Model Empowered Recommendation Approachcited by 168
  11. 2025CoT-Valve: Length-Compressible Chain-of-Thought Tuningcited by 167
  12. 2023How Can Recommender Systems Benefit from Large Language Models: A Surveycited by 141
  13. 2025Sketch-of-Thought: Efficient LLM Reasoning with Adaptive Cognitive-Inspired Sketchingcited by 116
  14. 2023A Bi-Step Grounding Paradigm for Large Language Models in Recommendation Systemscited by 73
  15. 2025Unlocking Efficient Long-to-Short LLM Reasoning with Model Mergingcited by 58
  16. 2023LLaRA: Large Language-Recommendation Assistantcited by 56
  17. 2025Activation-Informed Merging of Large Language Modelscited by 29
  18. 2024RDRec: Rationale Distillation for LLM-based Recommendationcited by 21
  19. 2025Sens-Merging: Sensitivity-Guided Parameter Balancing for Merging Large Language Modelscited by 16
  20. 2025Activation-Guided Consensus Merging for Large Language Modelscited by 12
  21. 2025Thinking Sparks!: Emergent Attention Heads in Reasoning Models During Post Trainingcited by 8
  22. 2026RAIN-Merging: A Gradient-Free Method to Enhance Instruction Following in Large Reasoning Models with Preserved Thinking Formatcited by 7
  23. 2025RCP-Merging: Merging Long Chain-of-Thought Models with Domain-Specific Models by Considering Reasoning Capability as Priorcited by 5
  24. 2025WeaveRec: An LLM-Based Cross-Domain Sequential Recommendation Framework with Model Mergingcited by 3
  25. 2023Fisher-Weighted Merge of Contrastive Learning Models in Sequential Recommendationcited by 2
  26. 2025Breaking the Aggregation Bottleneck in Federated Recommendation: A Personalized Model Merging Approachcited by 1
  27. 2026MMGRid: Navigating Temporal-aware and Cross-domain Generative Recommendation via Model Mergingcited by 1

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2026Reasoning to Rank: An End-to-End Solution for Exploiting Large Language Models for Recommendation
  2. 2025Reason-to-Recommend: Using Interaction-of-Thought Reasoning to Enhance LLM Recommendation
  3. 2025R2ec: Towards Large Recommender Models with Reasoning
  4. 2026SelfDR: Self-Distillation from Reasoning for LLM-Based Recommendation
  5. 2025Selective LLM-Guided Regularization for Enhancing Recommendation Models
  6. 2023Enhancing Recommender Systems with Large Language Model Reasoning Graphs
  7. 2026rEDMRec: Distilling Large Language Model Reasoning into an Editable Experience Memory for Recommendation
  8. 2026RRCM: Ranking-Driven Retrieval over Collaborative and Meta Memories for LLM Recommendation