# Bridge the Gap between Past and Future: Siamese Model Optimization for Context-Aware Document Ranking

arXiv: https://arxiv.org/abs/2505.14180 · HTML: https://mlsanity.com/paper/2505.14180/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 24 of its 47 references that are in the map.
- First posted on arXiv 20 May 2025.

## Builds on (most cited first, 24 of 24)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [PyTorch: An Imperative Style, High-Performance Deep Learning Library](https://mlsanity.com/paper/1912.01703/) (2019)
- [Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer](https://mlsanity.com/paper/1910.10683/) (2019)
- [Rethinking the Inception Architecture for Computer Vision](https://mlsanity.com/paper/1512.00567/) (2015)
- [Decoupled Weight Decay Regularization](https://mlsanity.com/paper/1711.05101/) (2017)
- [When Does Label Smoothing Help?](https://mlsanity.com/paper/1906.02629/) (2019)
- [Passage Re-ranking with BERT](https://mlsanity.com/paper/1901.04085/) (2019)
- [Document Ranking with a Pretrained Sequence-to-Sequence Model](https://mlsanity.com/paper/2003.06713/) (2020)
- [End-to-End Neural Ad-hoc Ranking with Kernel Pooling](https://mlsanity.com/paper/1706.06613/) (2017)
- [Convolutional Neural Network Architectures for Matching Natural Language Sentences](https://mlsanity.com/paper/1503.03244/) (2015)
- [Learning to Match Using Local and Distributed Representations of Text for Web Search](https://mlsanity.com/paper/1610.08136/) (2016)
- [Pytrec_eval: An Extremely Fast Python Interface to trec_eval](https://mlsanity.com/paper/1805.01597/) (2018)
- [Diversifying Dialog Generation via Adaptive Label Smoothing](https://mlsanity.com/paper/2105.14556/) (2021)
- [Asynchronous Bidirectional Decoding for Neural Machine Translation](https://mlsanity.com/paper/1801.05122/) (2018)
- [Privileged Features Distillation at Taobao Recommendations](https://mlsanity.com/paper/1907.05171/) (2019)
- [Contrastive Learning of User Behavior Sequence for Context-Aware Document Ranking](https://mlsanity.com/paper/2108.10510/) (2021)
- [Regularizing Dialogue Generation by Imitating Implicit Scenarios](https://mlsanity.com/paper/2010.01893/) (2020)
- [Context Attentive Document Ranking and Query Suggestion](https://mlsanity.com/paper/1906.02329/) (2019)
- [Posterior-GAN: Towards Informative and Coherent Response Generation with Posterior Generative Adversarial Network](https://mlsanity.com/paper/2003.02020/) (2020)
- [DialoGPS: Dialogue Path Sampling in Continuous Semantic Space for Data Augmentation in Multi-Turn Conversations](https://mlsanity.com/paper/2306.16770/) (2023)
- [From Easy to Hard: A Dual Curriculum Learning Framework for Context-Aware Document Ranking](https://mlsanity.com/paper/2208.10226/) (2022)
- [Enhancing User Behavior Sequence Modeling by Generative Tasks for Session Search](https://mlsanity.com/paper/2208.10846/) (2022)
- [Twin Networks: Matching the Future for Sequence Generation](https://mlsanity.com/paper/1708.06742/) (2017)

## Abstract (arXiv)

In the realm of information retrieval, users often engage in multi-turn interactions with search engines to acquire information, leading to the formation of sequences of user feedback behaviors. Leveraging the session context has proven to be beneficial for inferring user search intent and document ranking. A multitude of approaches have been proposed to exploit in-session context for improved document ranking. Despite these advances, the limitation of historical session data for capturing evolving user intent remains a challenge. In this work, we explore the integration of future contextual information into the session context to enhance document ranking. We present the siamese model optimization framework, comprising a history-conditioned model and a future-aware model. The former processes only the historical behavior sequence, while the latter integrates both historical and anticipated future behaviors. Both models are trained collaboratively using the supervised labels and pseudo labels predicted by the other. The history-conditioned model, referred to as ForeRanker, progressively learns future-relevant information to enhance ranking, while it singly uses historical session at inference time. To mitigate inconsistencies during training, we introduce the peer knowledge distillation method with a dynamic gating mechanism, allowing models to selectively incorporate contextual information. Experimental results on benchmark datasets demonstrate the effectiveness of our ForeRanker, showcasing its superior performance compared to existing methods.