# IROF: a low resource evaluation metric for explanation methods

arXiv: https://arxiv.org/abs/2003.08747 · HTML: https://mlsanity.com/paper/2003.08747/

## Key facts

- Cited by 46 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 9 Mar 2020.

## Built on by (most cited first, 30 of 46)

- [Quantus: An Explainable AI Toolkit for Responsible Evaluation of Neural Network Explanations and Beyond](https://mlsanity.com/paper/2202.06861/) (2022)
- [The Meta-Evaluation Problem in Explainable AI: Identifying Reliable Estimators with MetaQuantus](https://mlsanity.com/paper/2302.07265/) (2023)
- [PEDANTS: Cheap but Effective and Interpretable Answer Equivalence](https://mlsanity.com/paper/2402.11161/) (2024)
- [Decoupling Pixel Flipping and Occlusion Strategy for Consistent XAI Benchmarks](https://mlsanity.com/paper/2401.06654/) (2024)
- [Finding the right XAI method -- A Guide for the Evaluation and Ranking of Explainable AI Methods in Climate Science](https://mlsanity.com/paper/2303.00652/) (2023)
- [Xplique: A Deep Learning Explainability Toolbox](https://mlsanity.com/paper/2206.04394/) (2022)
- [Evaluating Feature Attribution Methods in the Image Domain](https://mlsanity.com/paper/2202.12270/) (2022)
- [A comprehensive study on fidelity metrics for XAI](https://mlsanity.com/paper/2401.10640/) (2024)
- [Navigating the Maze of Explainable AI: A Systematic Approach to Evaluating Methods and Metrics](https://mlsanity.com/paper/2409.16756/) (2024)
- [Assessing Fidelity in XAI post-hoc techniques: A Comparative Study with Ground Truth Explanations Datasets](https://mlsanity.com/paper/2311.01961/) (2023)
- [RELAX: Representation Learning Explainability](https://mlsanity.com/paper/2112.10161/) (2021)
- [Feature Perturbation Augmentation for Reliable Evaluation of Importance Estimators in Neural Networks](https://mlsanity.com/paper/2303.01538/) (2023)
- [Sanity Checks Revisited: An Exploration to Repair the Model Parameter Randomisation Test](https://mlsanity.com/paper/2401.06465/) (2024)
- [Causal Analysis for Robust Interpretability of Neural Networks](https://mlsanity.com/paper/2305.08950/) (2023)
- [A Fresh Look at Sanity Checks for Saliency Maps](https://mlsanity.com/paper/2405.02383/) (2024)
- [Benchmarking the Attribution Quality of Vision Models](https://mlsanity.com/paper/2407.11910/) (2024)
- [Benchmarking XAI Explanations with Human-Aligned Evaluations](https://mlsanity.com/paper/2411.02470/) (2024)
- [An adversarial attack approach for eXplainable AI evaluation on deepfake detection models](https://mlsanity.com/paper/2312.06627/) (2023)
- [Feature Attribution with Necessity and Sufficiency via Dual-stage Perturbation Test for Causal Explanation](https://mlsanity.com/paper/2402.08845/) (2024)
- [On the Robustness of Explanations of Deep Neural Network Models: A Survey](https://mlsanity.com/paper/2211.04780/) (2022)
- [Expected Grad-CAM: Towards gradient faithfulness](https://mlsanity.com/paper/2406.01274/) (2024)
- [From Flexibility to Manipulation: The Slippery Slope of XAI Evaluation](https://mlsanity.com/paper/2412.05592/) (2024)
- [Spatio-Temporal Perturbations for Video Attribution](https://mlsanity.com/paper/2109.00222/) (2021)
- [Robustness of Visual Explanations to Common Data Augmentation](https://mlsanity.com/paper/2304.08984/) (2023)
- [On the Effectiveness of Methods and Metrics for Explainable AI in Remote Sensing Image Scene Classification](https://mlsanity.com/paper/2507.05916/) (2025)
- [A Robust Unsupervised Ensemble of Feature-Based Explanations using Restricted Boltzmann Machines](https://mlsanity.com/paper/2111.07379/) (2021)
- [Fidelity of Ensemble Aggregation for Saliency Map Explanations using Bayesian Optimization Techniques](https://mlsanity.com/paper/2207.01565/) (2022)
- [Advancing Attribution-Based Neural Network Explainability through Relative Absolute Magnitude Layer-Wise Relevance Propagation and Multi-Component Evaluation](https://mlsanity.com/paper/2412.09311/) (2024)
- [Reconciling Privacy and Explainability in High-Stakes: A Systematic Inquiry](https://mlsanity.com/paper/2412.20798/) (2024)
- [Fast Fourier Correlation is a Highly Efficient and Accurate Feature Attribution Algorithm from the Perspective of Control Theory and Game Theory](https://mlsanity.com/paper/2504.02016/) (2025)

## Abstract (arXiv)

The adoption of machine learning in health care hinges on the transparency of the used algorithms, necessitating the need for explanation methods. However, despite a growing literature on explaining neural networks, no consensus has been reached on how to evaluate those explanation methods. We propose IROF, a new approach to evaluating explanation methods that circumvents the need for manual evaluation. Compared to other recent work, our approach requires several orders of magnitude less computational resources and no human input, making it accessible to lower resource groups and robust to human bias.