# RankSHAP: Shapley Value Based Feature Attributions for Learning to Rank

arXiv: https://arxiv.org/abs/2405.01848 · HTML: https://mlsanity.com/paper/2405.01848/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- Builds on 25 of its 49 references that are in the map.
- First posted on arXiv 3 May 2024.

## Builds on (most cited first, 25 of 25)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer](https://mlsanity.com/paper/1910.10683/) (2019)
- [Llama 2: Open Foundation and Fine-Tuned Chat Models](https://mlsanity.com/paper/2307.09288/) (2023)
- [A Unified Approach to Interpreting Model Predictions](https://mlsanity.com/paper/1705.07874/) (2017)
- ["Why Should I Trust You?": Explaining the Predictions of Any Classifier](https://mlsanity.com/paper/1602.04938/) (2016)
- [Learning Important Features Through Propagating Activation Differences](https://mlsanity.com/paper/1704.02685/) (2017)
- [Passage Re-ranking with BERT](https://mlsanity.com/paper/1901.04085/) (2019)
- [Document Ranking with a Pretrained Sequence-to-Sequence Model](https://mlsanity.com/paper/2003.06713/) (2020)
- [Fine-Tuning LLaMA for Multi-Stage Text Retrieval](https://mlsanity.com/paper/2310.08319/) (2023)
- [PRADA: Practical Black-Box Adversarial Attacks against Neural Ranking Models](https://mlsanity.com/paper/2204.01321/) (2022)
- [A Weighted Correlation Index for Rankings with Ties](https://mlsanity.com/paper/1404.3325/) (2014)
- [NeuralNDCG: Direct Optimisation of a Ranking Metric via Differentiable Relaxation of Sorting](https://mlsanity.com/paper/2102.07831/) (2021)
- [Feature Importance Ranking for Deep Learning](https://mlsanity.com/paper/2010.08973/) (2020)
- [Explain and Predict, and then Predict Again](https://mlsanity.com/paper/2101.04109/) (2021)
- [One word at a time: adversarial attacks on retrieval models](https://mlsanity.com/paper/2008.02197/) (2020)
- [A study on the Interpretability of Neural Retrieval Models using DeepSHAP](https://mlsanity.com/paper/1907.06484/) (2019)
- [EXS: Explainable Search Using Local Model Agnostic Interpretability](https://mlsanity.com/paper/1809.03857/) (2018)
- [Explainable Information Retrieval: A Survey](https://mlsanity.com/paper/2211.02405/) (2022)
- [Towards Axiomatic Explanations for Neural Ranking Models](https://mlsanity.com/paper/2106.08019/) (2021)
- [Rank-LIME: Local Model-Agnostic Feature Attribution for Learning to Rank](https://mlsanity.com/paper/2212.12722/) (2022)
- [Explaining Preferences with Shapley Values](https://mlsanity.com/paper/2205.13662/) (2022)
- [Explaining Documents' Relevance to Search Queries](https://mlsanity.com/paper/2111.01314/) (2021)
- [Valid Explanations for Learning to Rank Models](https://mlsanity.com/paper/2004.13972/) (2020)
- [Finding Inverse Document Frequency Information in BERT](https://mlsanity.com/paper/2202.12191/) (2022)
- [Interpretable Learning-to-Rank with Generalized Additive Models](https://mlsanity.com/paper/2005.02553/) (2020)

## Built on by (most cited first, 4 of 4)

- [LLM-Confidence Reranker: A Training-Free Approach for Enhancing Retrieval-Augmented Generation Systems](https://mlsanity.com/paper/2602.13571/) (2026)
- [Uncertainty-Aware Multimodal Learning via Conformal Shapley Intervals](https://mlsanity.com/paper/2602.00171/) (2026)
- [CQD-SHAP: Explainable Complex Query Answering via Shapley Values](https://mlsanity.com/paper/2510.15623/) (2025)
- [Population-Level Generative Modeling for Ranking Data](https://mlsanity.com/paper/2608.08422/) (2026)

## Abstract (arXiv)

Numerous works propose post-hoc, model-agnostic explanations for learning to rank, focusing on ordering entities by their relevance to a query through feature attribution methods. However, these attributions often weakly correlate or contradict each other, confusing end users. We adopt an axiomatic game-theoretic approach, popular in the feature attribution community, to identify a set of fundamental axioms that every ranking-based feature attribution method should satisfy. We then introduce Rank-SHAP, extending classical Shapley values to ranking. We evaluate the RankSHAP framework through extensive experiments on two datasets, multiple ranking methods and evaluation metrics. Additionally, a user study confirms RankSHAP's alignment with human intuition. We also perform an axiomatic analysis of existing rank attribution algorithms to determine their compliance with our proposed axioms. Ultimately, our aim is to equip practitioners with a set of axiomatically backed feature attribution methods for studying IR ranking models, that ensure generality as well as consistency.