# Evaluating Local Model-Agnostic Explanations of Learning to Rank Models with Decision Paths

arXiv: https://arxiv.org/abs/2203.02295 · HTML: https://mlsanity.com/paper/2203.02295/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 11 of its 31 references that are in the map.
- First posted on arXiv 4 Mar 2022.

## Builds on (most cited first, 11 of 11)

- [XGBoost: A Scalable Tree Boosting System](https://mlsanity.com/paper/1603.02754/) (2016)
- ["Why Should I Trust You?": Explaining the Predictions of Any Classifier](https://mlsanity.com/paper/1602.04938/) (2016)
- [Towards A Rigorous Science of Interpretable Machine Learning](https://mlsanity.com/paper/1702.08608/) (2017)
- [Interpretation of Neural Networks is Fragile](https://mlsanity.com/paper/1710.10547/) (2017)
- [Methods for Interpreting and Understanding Deep Neural Networks](https://mlsanity.com/paper/1706.07979/) (2017)
- [Evaluations and Methods for Explanation through Robustness Analysis](https://mlsanity.com/paper/2006.00442/) (2020)
- [Looking Deeper into Tabular LIME](https://mlsanity.com/paper/2008.11092/) (2020)
- [Visual Question Answering using Deep Learning: A Survey and Performance Analysis](https://mlsanity.com/paper/1909.01860/) (2019)
- [EXS: Explainable Search Using Local Model Agnostic Interpretability](https://mlsanity.com/paper/1809.03857/) (2018)
- [Pairwise Learning to Rank by Neural Networks Revisited: Reconstruction, Theoretical Analysis and Practical Performance](https://mlsanity.com/paper/1909.02768/) (2019)
- [Explainable Machine Learning for Fraud Detection](https://mlsanity.com/paper/2105.06314/) (2021)

## Abstract (arXiv)

Local explanations of learning-to-rank (LTR) models are thought to extract the most important features that contribute to the ranking predicted by the LTR model for a single data point. Evaluating the accuracy of such explanations is challenging since the ground truth feature importance scores are not available for most modern LTR models. In this work, we propose a systematic evaluation technique for explanations of LTR models. Instead of using black-box models, such as neural networks, we propose to focus on tree-based LTR models, from which we can extract the ground truth feature importance scores using decision paths. Once extracted, we can directly compare the ground truth feature importance scores to the feature importance scores generated with explanation techniques. We compare two recently proposed explanation techniques for LTR models when using decision trees and gradient boosting models on the MQ2008 dataset. We show that the explanation accuracy in these techniques can largely vary depending on the explained model and even which data point is explained.