# From Top-1 to Top-K: A Reproducibility Study and Benchmarking of Counterfactual Explanations for Recommender Systems

arXiv: https://arxiv.org/abs/2604.19663 · HTML: https://mlsanity.com/paper/2604.19663/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 17 of its 40 references that are in the map.
- First posted on arXiv 21 Apr 2026.

## Builds on (most cited first, 17 of 17)

- ["Why Should I Trust You?": Explaining the Predictions of Any Classifier](https://mlsanity.com/paper/1602.04938/) (2016)
- [Understanding Black-box Predictions via Influence Functions](https://mlsanity.com/paper/1703.04730/) (2017)
- [LightGCN: Simplifying and Powering Graph Convolution Network for Recommendation](https://mlsanity.com/paper/2002.02126/) (2020)
- [Variational Autoencoders for Collaborative Filtering](https://mlsanity.com/paper/1802.05814/) (2018)
- [Explainable Recommendation: A Survey and New Perspectives](https://mlsanity.com/paper/1804.11192/) (2018)
- [Are Graph Augmentations Necessary? Simple Graph Contrastive Learning for Recommendation](https://mlsanity.com/paper/2112.08679/) (2021)
- [Diffusion Recommender Model](https://mlsanity.com/paper/2304.04971/) (2023)
- [Learning and Evaluating Graph Neural Network Explanations based on Counterfactual and Factual Reasoning](https://mlsanity.com/paper/2202.08816/) (2022)
- [CLEAR: Generative Counterfactual Explanations on Graphs](https://mlsanity.com/paper/2210.08443/) (2022)
- [PRINCE: Provider-side Interpretability with Counterfactual Explanations in Recommender Systems](https://mlsanity.com/paper/1911.08378/) (2019)
- [Graph Transformer for Recommendation](https://mlsanity.com/paper/2306.02330/) (2023)
- [Counterfactual Explanations for Neural Recommenders](https://mlsanity.com/paper/2105.05008/) (2021)
- [GREASE: Generate Factual and Counterfactual Explanations for GNN-based Recommendations](https://mlsanity.com/paper/2208.04222/) (2022)
- [Explaining Latent Factor Models for Recommendation with Influence Functions](https://mlsanity.com/paper/1811.08120/) (2018)
- [Counterfactually Evaluating Explanations in Recommender Systems](https://mlsanity.com/paper/2203.01310/) (2022)
- [A Reproducibility Study of Product-side Fairness in Bundle Recommendation](https://mlsanity.com/paper/2507.14352/) (2025)
- [Explaining Group Recommendations via Counterfactuals](https://mlsanity.com/paper/2601.16882/) (2026)

## Abstract (arXiv)

Counterfactual explanations (CEs) provide an intuitive way to understand recommender systems by identifying minimal modifications to user-item interactions that alter recommendation outcomes. Existing CE methods for recommender systems, however, have been evaluated under heterogeneous protocols, using different datasets, recommenders, metrics, and even explanation formats, which hampers reproducibility and fair comparison. Our paper systematically reproduces, re-implement, and re-evaluate eleven state-of-the-art CE methods for recommender systems, covering both native explainers (e.g., LIME-RS, SHAP, PRINCE, ACCENT, LXR, GREASE) and specific graph-based explainers originally proposed for GNNs. Here, a unified benchmarking framework is proposed to assess explainers along three dimensions: explanation format (implicit vs. explicit), evaluation level (item-level vs. list-level), and perturbation scope (user interaction vectors vs. user-item interaction graphs). Our evaluation protocol includes effectiveness, sparsity, and computational complexity metrics, and extends existing item-level assessments to top-K list-level explanations. Through extensive experiments on three real-world datasets and six representative recommender models, we analyze how well previously reported strengths of CE methods generalize across diverse setups. We observe that the trade-off between effectiveness and sparsity depends strongly on the specific method and evaluation setting, particularly under the explicit format; in addition, explainer performance remains largely consistent across item level and list level evaluations, and several graph-based explainers exhibit notable scalability limitations on large recommender graphs. Our results refine and challenge earlier conclusions about the robustness and practicality of CE generation methods in recommender systems: https://github.com/L2R-UET/CFExpRec.