# Temporal Information Retrieval via Time-Specifier Model Merging

arXiv: https://arxiv.org/abs/2507.06782 · HTML: https://mlsanity.com/paper/2507.06782/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- Builds on 27 of its 36 references that are in the map.
- First posted on arXiv 9 Jul 2025.

## Builds on (most cited first, 27 of 27)

- [LoRA: Low-Rank Adaptation of Large Language Models](https://mlsanity.com/paper/2106.09685/) (2021)
- [Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks](https://mlsanity.com/paper/2005.11401/) (2020)
- [Overcoming catastrophic forgetting in neural networks](https://mlsanity.com/paper/1612.00796/) (2016)
- [Dense Passage Retrieval for Open-Domain Question Answering](https://mlsanity.com/paper/2004.04906/) (2020)
- [Learning without Forgetting](https://mlsanity.com/paper/1606.09282/) (2016)
- [Decoupled Weight Decay Regularization](https://mlsanity.com/paper/1711.05101/) (2017)
- [MS MARCO: A Human Generated MAchine Reading COmprehension Dataset](https://mlsanity.com/paper/1611.09268/) (2016)
- [BEIR: A Heterogenous Benchmark for Zero-shot Evaluation of Information Retrieval Models](https://mlsanity.com/paper/2104.08663/) (2021)
- [Unsupervised Dense Information Retrieval with Contrastive Learning](https://mlsanity.com/paper/2112.09118/) (2021)
- [Generalization through Memorization: Nearest Neighbor Language Models](https://mlsanity.com/paper/1911.00172/) (2019)
- [An Empirical Investigation of Catastrophic Forgetting in Gradient-Based Neural Networks](https://mlsanity.com/paper/1312.6211/) (2013)
- [An Empirical Study of Catastrophic Forgetting in Large Language Models During Continual Fine-tuning](https://mlsanity.com/paper/2308.08747/) (2023)
- [Time-Aware Language Models as Temporal Knowledge Bases](https://mlsanity.com/paper/2106.15110/) (2021)
- [Ensemble deep learning: A review](https://mlsanity.com/paper/2104.02395/) (2021)
- [Model Merging in LLMs, MLLMs, and Beyond: Methods, Theories, Applications and Opportunities](https://mlsanity.com/paper/2408.07666/) (2024)
- [Arcee's MergeKit: A Toolkit for Merging Large Language Models](https://mlsanity.com/paper/2403.13257/) (2024)
- [A Dataset for Answering Time-Sensitive Questions](https://mlsanity.com/paper/2108.06314/) (2021)
- [Encoder Based Lifelong Learning](https://mlsanity.com/paper/1704.01920/) (2017)
- [Multi-passage BERT: A Globally Normalized BERT Model for Open-domain Question Answering](https://mlsanity.com/paper/1908.08167/) (2019)
- [Model Stock: All we need is just a few fine-tuned models](https://mlsanity.com/paper/2403.19522/) (2024)
- [A Survey on Complex Knowledge Base Question Answering: Methods, Challenges and Solutions](https://mlsanity.com/paper/2105.11644/) (2021)
- [Time Masking for Temporal Language Models](https://mlsanity.com/paper/2110.06366/) (2021)
- [LM-Cocktail: Resilient Tuning of Language Models via Model Merging](https://mlsanity.com/paper/2311.13534/) (2023)
- [Mitigating Catastrophic Forgetting in Language Transfer via Model Merging](https://mlsanity.com/paper/2407.08699/) (2024)
- [BiTimeBERT: Extending Pre-Trained Language Representations with Bi-Temporal Information](https://mlsanity.com/paper/2204.13032/) (2022)
- [TempRetriever: Fusion-based Temporal Dense Passage Retrieval for Time-Sensitive Questions](https://mlsanity.com/paper/2502.21024/) (2025)
- [Salient Span Masking for Temporal Understanding](https://mlsanity.com/paper/2303.12860/) (2023)

## Built on by (most cited first, 5 of 5)

- [Taxonomy of the Retrieval System Framework: Pitfalls and Paradigms](https://mlsanity.com/paper/2601.20131/) (2026)
- [Efficient Temporal-aware Matryoshka Adaptation for Temporal Information Retrieval](https://mlsanity.com/paper/2601.05549/) (2026)
- [Re3: Learning to Balance Relevance & Recency for Temporal Information Retrieval](https://mlsanity.com/paper/2509.01306/) (2025)
- [ChunQiuTR: Time-Keyed Temporal Retrieval in Classical Chinese Annals](https://mlsanity.com/paper/2604.06997/) (2026)
- [Q-TIE: A Lightweight and Generalizable Re-ranking Framework for Temporal Information Retrieval](https://mlsanity.com/paper/2609.23880/) (2026)

## Abstract (arXiv)

The rapid expansion of digital information and knowledge across structured and unstructured sources has heightened the importance of Information Retrieval (IR). While dense retrieval methods have substantially improved semantic matching for general queries, they consistently underperform on queries with explicit temporal constraints--often those containing numerical expressions and time specifiers such as ``in 2015.'' Existing approaches to Temporal Information Retrieval (TIR) improve temporal reasoning but often suffer from catastrophic forgetting, leading to reduced performance on non-temporal queries. To address this, we propose Time-Specifier Model Merging (TSM), a novel method that enhances temporal retrieval while preserving accuracy on non-temporal queries. TSM trains specialized retrievers for individual time specifiers and merges them in to a unified model, enabling precise handling of temporal constraints without compromising non-temporal retrieval. Extensive experiments on both temporal and non-temporal datasets demonstrate that TSM significantly improves performance on temporally constrained queries while maintaining strong results on non-temporal queries, consistently outperforming other baseline methods. Our code is available at https://github.com/seungyoonee/TSM .