# Understanding Matching Mechanisms in Cross-Encoders

arXiv: https://arxiv.org/abs/2507.14604 · HTML: https://mlsanity.com/paper/2507.14604/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 7 of its 10 references that are in the map.
- First posted on arXiv 19 Jul 2025.

## Builds on (most cited first, 7 of 7)

- [Passage Re-ranking with BERT](https://mlsanity.com/paper/1901.04085/) (2019)
- [Overview of the TREC 2019 deep learning track](https://mlsanity.com/paper/2003.07820/) (2020)
- [Overview of the TREC 2020 deep learning track](https://mlsanity.com/paper/2102.07662/) (2021)
- [Restricting the Flow: Information Bottlenecks for Attribution](https://mlsanity.com/paper/2001.00396/) (2020)
- [Overview of the TREC 2022 deep learning track](https://mlsanity.com/paper/2507.10865/) (2025)
- [Overview of the TREC 2021 deep learning track](https://mlsanity.com/paper/2507.08191/) (2025)
- [Axiomatic Causal Interventions for Reverse Engineering Relevance Computation in Neural Retrieval Models](https://mlsanity.com/paper/2405.02503/) (2024)

## Built on by (most cited first, 1 of 1)

- [Cross-Attention Calibrated Deduplication for Retrieval-Augmented Generation System](https://mlsanity.com/paper/2607.24332/) (2026)

## Abstract (arXiv)

Neural IR architectures, particularly cross-encoders, are highly effective models whose internal mechanisms are mostly unknown. Most works trying to explain their behavior focused on high-level processes (e.g., what in the input influences the prediction, does the model adhere to known IR axioms) but fall short of describing the matching process. Instead of Mechanistic Interpretability approaches which specifically aim at explaining the hidden mechanisms of neural models, we demonstrate that more straightforward methods can already provide valuable insights. In this paper, we first focus on the attention process and extract causal insights highlighting the crucial roles of some attention heads in this process. Second, we provide an interpretation of the mechanism underlying matching detection.