# Improving Molecular Graph Neural Network Explainability with Orthonormalization and Induced Sparsity

arXiv: https://arxiv.org/abs/2105.04854 · HTML: https://mlsanity.com/paper/2105.04854/

## Key facts

- Cited by 20 papers in the map (data through 14 Oct 2026).
- Builds on 22 of its 47 references that are in the map.
- First posted on arXiv 11 May 2021.

## Builds on (most cited first, 22 of 22)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Grad-CAM: Visual Explanations from Deep Networks via Gradient-based Localization](https://mlsanity.com/paper/1610.02391/) (2016)
- [Axiomatic Attribution for Deep Networks](https://mlsanity.com/paper/1703.01365/) (2017)
- [Learning Deep Features for Discriminative Localization](https://mlsanity.com/paper/1512.04150/) (2015)
- [Fast Graph Representation Learning with PyTorch Geometric](https://mlsanity.com/paper/1903.02428/) (2019)
- [Learning Important Features Through Propagating Activation Differences](https://mlsanity.com/paper/1704.02685/) (2017)
- [SmoothGrad: removing noise by adding noise](https://mlsanity.com/paper/1706.03825/) (2017)
- [Deep Graph Infomax](https://mlsanity.com/paper/1809.10341/) (2018)
- [Convolutional Networks on Graphs for Learning Molecular Fingerprints](https://mlsanity.com/paper/1509.09292/) (2015)
- [GNNExplainer: Generating Explanations for Graph Neural Networks](https://mlsanity.com/paper/1903.03894/) (2019)
- [InfoGraph: Unsupervised and Semi-supervised Graph-Level Representation Learning via Mutual Information Maximization](https://mlsanity.com/paper/1908.01000/) (2019)
- [Unitary Evolution Recurrent Neural Networks](https://mlsanity.com/paper/1511.06464/) (2015)
- [Graph Information Bottleneck](https://mlsanity.com/paper/2010.12811/) (2020)
- [Interpretable Convolutional Neural Networks](https://mlsanity.com/paper/1710.00935/) (2017)
- [Explainability Techniques for Graph Convolutional Networks](https://mlsanity.com/paper/1905.13686/) (2019)
- [Interpreting Deep Visual Representations via Network Dissection](https://mlsanity.com/paper/1711.05611/) (2017)
- [Orthogonal Weight Normalization: Solution to Optimization over Multiple Dependent Stiefel Manifolds in Deep Neural Networks](https://mlsanity.com/paper/1709.06079/) (2017)
- [On orthogonality and learning recurrent networks with long term dependencies](https://mlsanity.com/paper/1702.00071/) (2017)
- [Tunable Efficient Unitary Neural Networks (EUNN) and their application to RNNs](https://mlsanity.com/paper/1612.05231/) (2016)
- [All You Need is Beyond a Good Init: Exploring Better Solution for Training Extremely Deep Convolutional Neural Networks with Orthonormality and Modulation](https://mlsanity.com/paper/1703.01827/) (2017)
- [Interpreting and Understanding Graph Convolutional Neural Network using Gradient-based Attribution Method](https://mlsanity.com/paper/1903.03768/) (2019)
- [A comprehensive study on the prediction reliability of graph neural networks for virtual screening](https://mlsanity.com/paper/2003.07611/) (2020)

## Built on by (most cited first, 20 of 20)

- [Interpretable and Generalizable Graph Learning via Stochastic Attention Mechanism](https://mlsanity.com/paper/2201.12987/) (2022)
- [Trustworthy Graph Neural Networks: Aspects, Methods and Trends](https://mlsanity.com/paper/2205.07424/) (2022)
- [Graph-based Molecular Representation Learning](https://mlsanity.com/paper/2207.04869/) (2022)
- [Encoding Time-Series Explanations through Self-Supervised Model Behavior Consistency](https://mlsanity.com/paper/2306.02109/) (2023)
- [A Survey on Explainability of Graph Neural Networks](https://mlsanity.com/paper/2306.01958/) (2023)
- [A Survey of Trustworthy Graph Learning: Reliability, Explainability, and Privacy Protection](https://mlsanity.com/paper/2205.10014/) (2022)
- [Learning the Solution Operator of Boundary Value Problems using Graph Neural Networks](https://mlsanity.com/paper/2206.14092/) (2022)
- [MEGAN: Multi-Explanation Graph Attention Network](https://mlsanity.com/paper/2211.13236/) (2022)
- [Explaining, Evaluating and Enhancing Neural Networks' Learned Representations](https://mlsanity.com/paper/2202.09374/) (2022)
- [Graph AI in Medicine](https://mlsanity.com/paper/2310.13767/) (2023)
- [Recent Developments in GNNs for Drug Discovery](https://mlsanity.com/paper/2506.01302/) (2025)
- [Physics-Informed Graph Learning](https://mlsanity.com/paper/2202.10679/) (2022)
- [Advancements in Molecular Property Prediction: A Survey of Single and Multimodal Approaches](https://mlsanity.com/paper/2408.09461/) (2024)
- [Recent advances in interpretable machine learning using structure-based protein representations](https://mlsanity.com/paper/2409.17726/) (2024)
- [Fragment-Wise Interpretability in Graph Neural Networks via Molecule Decomposition and Contribution Analysis](https://mlsanity.com/paper/2508.15015/) (2025)
- [Towards Learning Self-Organized Criticality of Rydberg Atoms using Graph Neural Networks](https://mlsanity.com/paper/2207.08927/) (2022)
- [Streamlining models with explanations in the learning loop](https://mlsanity.com/paper/2302.07760/) (2023)
- [ARM-Explainer -- Explaining and improving graph neural network predictions for the maximum clique problem using node features and association rule mining](https://mlsanity.com/paper/2511.22866/) (2025)
- [Archetypal Graph Generative Models: Explainable and Identifiable Communities via Anchor-Dominant Convex Hulls](https://mlsanity.com/paper/2602.21342/) (2026)
- [TAHB: A Comprehensive Benchmark for Text-Attributed Hypergraph Learning](https://mlsanity.com/paper/2608.15055/) (2026)

## Abstract (arXiv)

Rationalizing which parts of a molecule drive the predictions of a molecular graph convolutional neural network (GCNN) can be difficult. To help, we propose two simple regularization techniques to apply during the training of GCNNs: Batch Representation Orthonormalization (BRO) and Gini regularization. BRO, inspired by molecular orbital theory, encourages graph convolution operations to generate orthonormal node embeddings. Gini regularization is applied to the weights of the output layer and constrains the number of dimensions the model can use to make predictions. We show that Gini and BRO regularization can improve the accuracy of state-of-the-art GCNN attribution methods on artificial benchmark datasets. In a real-world setting, we demonstrate that medicinal chemists significantly prefer explanations extracted from regularized models. While we only study these regularizers in the context of GCNNs, both can be applied to other types of neural networks