# MMGRid: Navigating Temporal-aware and Cross-domain Generative Recommendation via Model Merging

arXiv: https://arxiv.org/abs/2601.15930 · HTML: https://mlsanity.com/paper/2601.15930/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 26 of its 43 references that are in the map.
- First posted on arXiv 22 Jan 2026.

## Builds on (most cited first, 26 of 26)

- [Scaling Laws for Neural Language Models](https://mlsanity.com/paper/2001.08361/) (2020)
- [Neural Discrete Representation Learning](https://mlsanity.com/paper/1711.00937/) (2017)
- [Model soups: averaging weights of multiple fine-tuned models improves accuracy without increasing inference time](https://mlsanity.com/paper/2203.05482/) (2022)
- [Self-Attentive Sequential Recommendation](https://mlsanity.com/paper/1808.09781/) (2018)
- [TIES-Merging: Resolving Interference When Merging Models](https://mlsanity.com/paper/2306.01708/) (2023)
- [Language Models are Super Mario: Absorbing Abilities from Homologous Models as a Free Lunch](https://mlsanity.com/paper/2311.03099/) (2023)
- [Evolutionary Optimization of Model Merging Recipes](https://mlsanity.com/paper/2403.13187/) (2024)
- [Adapting Large Language Models by Integrating Collaborative Semantics for Recommendation](https://mlsanity.com/paper/2311.09049/) (2023)
- [OneRec: Unifying Retrieve and Rank with Generative Recommender and Iterative Preference Alignment](https://mlsanity.com/paper/2502.18965/) (2025)
- [A Bi-Step Grounding Paradigm for Large Language Models in Recommendation Systems](https://mlsanity.com/paper/2308.08434/) (2023)
- [HLLM: Enhancing Sequential Recommendations via Hierarchical Large Language Models for Item and User Modeling](https://mlsanity.com/paper/2409.12740/) (2024)
- [A Critical Study on Data Leakage in Recommender System Offline Evaluation](https://mlsanity.com/paper/2010.11060/) (2020)
- [Sparse Meets Dense: Unified Generative Recommendations with Cascaded Sparse-Dense Representations](https://mlsanity.com/paper/2503.02453/) (2025)
- [Jointly Training Large Autoregressive Multimodal Models](https://mlsanity.com/paper/2309.15564/) (2023)
- [Revisiting Weight Averaging for Model Merging](https://mlsanity.com/paper/2412.12153/) (2024)
- [Order-agnostic Identifier for Large Language Model-based Generative Recommendation](https://mlsanity.com/paper/2502.10833/) (2025)
- [LLM2Rec: Large Language Models Are Powerful Embedding Models for Sequential Recommendation](https://mlsanity.com/paper/2506.21579/) (2025)
- [Semantic Convergence: Harmonizing Recommender Systems via Two-Stage Alignment and Behavioral Semantic Tokenization](https://mlsanity.com/paper/2412.13771/) (2024)
- [Diagnostic-Guided Dynamic Profile Optimization for LLM-based User Simulators in Sequential Recommendation](https://mlsanity.com/paper/2508.12645/) (2025)
- [Linear Combination of Saved Checkpoints Makes Consistency and Diffusion Models Better](https://mlsanity.com/paper/2404.02241/) (2024)
- [Generative Product Recommendations for Implicit Superlative Queries](https://mlsanity.com/paper/2504.18748/) (2025)
- [Enhancing New-item Fairness in Dynamic Recommender Systems](https://mlsanity.com/paper/2504.21362/) (2025)
- [Reinforcement Speculative Decoding for Fast Ranking](https://mlsanity.com/paper/2505.20316/) (2025)
- [Dual-Phase Playtime-guided Recommendation: Interest Intensity Exploration and Multimodal Random Walks](https://mlsanity.com/paper/2508.14058/) (2025)
- [PCR-CA: Parallel Codebook Representations with Contrastive Alignment for Multiple-Category App Recommendation](https://mlsanity.com/paper/2508.18166/) (2025)
- [RecBase: Generative Foundation Model Pretraining for Zero-Shot Recommendation](https://mlsanity.com/paper/2509.03131/) (2025)

## Built on by (most cited first, 1 of 1)

- [Towards Efficient Reasoning in LLM-Based Recommender Systems via Model Merging](https://mlsanity.com/paper/2608.10447/) (2026)

## Abstract (arXiv)

Model merging (MM) offers an efficient mechanism for integrating multiple specialized models without access to original training data or costly retraining. While MM has demonstrated success in domains like computer vision, its role in recommender systems (RSs) remains largely unexplored. Recently, Generative Recommendation (GR) has emerged as a new paradigm in RSs, characterized by rapidly growing model scales and substantial computational costs, making MM particularly appealing for cost-sensitive deployment scenarios. In this work, we present the first systematic study of MM in GR through a contextual lens. We focus on a fundamental yet underexplored challenge in real-world: how to merge generative recommenders specialized to different real-world contexts, arising from temporal evolving user behaviors and heterogeneous application domains. To this end, we propose a unified framework MMGRid, a structured contextual grid of GR checkpoints that organizes models trained under diverse contexts induced by temporal evolution and domain diversity. All checkpoints are derived from a shared base LLM but fine-tuned on context-specific data, forming a realistic and controlled model space for systematically analyzing MM across GR paradigms and merging algorithms. Our investigation reveals several key insights. First, training GR models from LLMs can introduce parameter conflicts during merging due to token distribution shifts and objective disparities; such conflicts can be alleviated by disentangling task-aware and context-specific parameter changes via base model replacement. Second, incremental training across contexts induces recency bias, which can be effectively balanced through weighted contextual merging. Notably, we observe that optimal merging weights correlate with context-dependent interaction characteristics, offering practical guidance for weight selection in real-world deployments.