# RelEx: A Model-Agnostic Relational Model Explainer

arXiv: https://arxiv.org/abs/2006.00305 · HTML: https://mlsanity.com/paper/2006.00305/

## Key facts

- Cited by 53 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 30 May 2020.

## Built on by (most cited first, 30 of 53)

- [A Comprehensive Survey on Trustworthy Graph Neural Networks: Privacy, Robustness, Fairness, and Explainability](https://mlsanity.com/paper/2204.08570/) (2022)
- [Trustworthy Graph Neural Networks: Aspects, Methods and Trends](https://mlsanity.com/paper/2205.07424/) (2022)
- [GraphFramEx: Towards Systematic Evaluation of Explainability Methods for Graph Neural Networks](https://mlsanity.com/paper/2206.09677/) (2022)
- [A Survey on Graph-Based Deep Learning for Computational Histopathology](https://mlsanity.com/paper/2107.00272/) (2021)
- [Explaining the Explainers in Graph Neural Networks: a Comparative Study](https://mlsanity.com/paper/2210.15304/) (2022)
- [Counterfactual Graphs for Explainable Classification of Brain Networks](https://mlsanity.com/paper/2106.08640/) (2021)
- [MEG: Generating Molecular Counterfactual Explanations for Deep Graph Networks](https://mlsanity.com/paper/2104.08060/) (2021)
- [A Survey on Explainability of Graph Neural Networks](https://mlsanity.com/paper/2306.01958/) (2023)
- [A Survey of Explainable Graph Neural Networks: Taxonomy and Evaluation Metrics](https://mlsanity.com/paper/2207.12599/) (2022)
- [A Survey of Trustworthy Graph Learning: Reliability, Explainability, and Privacy Protection](https://mlsanity.com/paper/2205.10014/) (2022)
- [GNNShap: Scalable and Accurate GNN Explanation using Shapley Values](https://mlsanity.com/paper/2401.04829/) (2024)
- [Graph Neural Networks in Modern AI-aided Drug Discovery](https://mlsanity.com/paper/2506.06915/) (2025)
- [MotifExplainer: a Motif-based Graph Neural Network Explainer](https://mlsanity.com/paper/2202.00519/) (2022)
- [BAGEL: A Benchmark for Assessing Graph Neural Network Explanations](https://mlsanity.com/paper/2206.13983/) (2022)
- [Interpretable Sparsification of Brain Graphs: Better Practices and Effective Designs for Graph Neural Networks](https://mlsanity.com/paper/2306.14375/) (2023)
- [GNNX-BENCH: Unravelling the Utility of Perturbation-based GNN Explainers through In-depth Benchmarking](https://mlsanity.com/paper/2310.01794/) (2023)
- [Explaining Graph Neural Networks via Structure-aware Interaction Index](https://mlsanity.com/paper/2405.14352/) (2024)
- [Deconfounding to Explanation Evaluation in Graph Neural Networks](https://mlsanity.com/paper/2201.08802/) (2022)
- [GOAt: Explaining Graph Neural Networks via Graph Output Attribution](https://mlsanity.com/paper/2401.14578/) (2024)
- [Defending Against Backdoor Attack on Graph Nerual Network by Explainability](https://mlsanity.com/paper/2209.02902/) (2022)
- [A Survey on Explainable Artificial Intelligence for Cybersecurity](https://mlsanity.com/paper/2303.12942/) (2023)
- [Graph Neural Network Causal Explanation via Neural Causal Models](https://mlsanity.com/paper/2407.09378/) (2024)
- [Comprehensible Artificial Intelligence on Knowledge Graphs: A survey](https://mlsanity.com/paper/2404.03499/) (2024)
- [L2XGNN: Learning to Explain Graph Neural Networks](https://mlsanity.com/paper/2209.14402/) (2022)
- [MAGE: Model-Level Graph Neural Networks Explanations via Motif-based Graph Generation](https://mlsanity.com/paper/2405.12519/) (2024)
- [Recent Advances in Malware Detection: Graph Learning and Explainability](https://mlsanity.com/paper/2502.10556/) (2025)
- [GRAPHSHAP: Explaining Identity-Aware Graph Classifiers Through the Language of Motifs](https://mlsanity.com/paper/2202.08815/) (2022)
- [Efficient GNN Explanation via Learning Removal-based Attribution](https://mlsanity.com/paper/2306.05760/) (2023)
- [ENGAGE: Explanation Guided Data Augmentation for Graph Representation Learning](https://mlsanity.com/paper/2307.01053/) (2023)
- [EiG-Search: Generating Edge-Induced Subgraphs for GNN Explanation in Linear Time](https://mlsanity.com/paper/2405.01762/) (2024)

## Abstract (arXiv)

In recent years, considerable progress has been made on improving the interpretability of machine learning models. This is essential, as complex deep learning models with millions of parameters produce state of the art results, but it can be nearly impossible to explain their predictions. While various explainability techniques have achieved impressive results, nearly all of them assume each data instance to be independent and identically distributed (iid). This excludes relational models, such as Statistical Relational Learning (SRL), and the recently popular Graph Neural Networks (GNNs), resulting in few options to explain them. While there does exist one work on explaining GNNs, GNN-Explainer, they assume access to the gradients of the model to learn explanations, which is restrictive in terms of its applicability across non-differentiable relational models and practicality. In this work, we develop RelEx, a model-agnostic relational explainer to explain black-box relational models with only access to the outputs of the black-box. RelEx is able to explain any relational model, including SRL models and GNNs. We compare RelEx to the state-of-the-art relational explainer, GNN-Explainer, and relational extensions of iid explanation models and show that RelEx achieves comparable or better performance, while remaining model-agnostic.