# Set-Encoder: Permutation-Invariant Inter-Passage Attention for Listwise Passage Re-Ranking with Cross-Encoders

arXiv: https://arxiv.org/abs/2404.06912 · HTML: https://mlsanity.com/paper/2404.06912/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 10 Apr 2024.

## Built on by (most cited first, 5 of 5)

- [Exploring the Effectiveness of Multi-stage Fine-tuning for Cross-encoder Re-rankers](https://mlsanity.com/paper/2503.22672/) (2025)
- [Comparative Analysis of Neural Retriever-Reranker Pipelines for Retrieval-Augmented Generation over Knowledge Graphs in E-commerce Applications](https://mlsanity.com/paper/2602.22219/) (2025)
- [Shapley Context Pruning: A Cooperative Game Perspective for Context Reranking and Pruning](https://mlsanity.com/paper/2607.16209/) (2026)
- [Robustness of IR Models to Collection Growth](https://mlsanity.com/paper/2608.23419/) (2026)
- [A Reranker for Orchestrating Heterogeneous Speech and Text Retrievers](https://mlsanity.com/paper/2608.26194/) (2026)

## Abstract (arXiv)

Existing cross-encoder models can be categorized as pointwise, pairwise, or listwise. Pairwise and listwise models allow passage interactions, which typically makes them more effective than pointwise models but less efficient and less robust to input passage order permutations. To enable efficient permutation-invariant passage interactions during re-ranking, we propose a new cross-encoder architecture with inter-passage attention: the Set-Encoder. In experiments on TREC Deep Learning and TIREx, the Set-Encoder is as effective as state-of-the-art listwise models while being more efficient and invariant to input passage order permutations. Compared to pointwise models, the Set-Encoder is particularly more effective when considering inter-passage information, such as novelty, and retains its advantageous properties compared to other listwise models. Our code is publicly available at https://github.com/webis-de/ECIR-25.