# Towards Hierarchical Importance Attribution: Explaining Compositional Semantics for Neural Sequence Models

arXiv: https://arxiv.org/abs/1911.06194 · HTML: https://mlsanity.com/paper/1911.06194/

## Key facts

- Cited by 26 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 8 Nov 2019.

## Built on by (most cited first, 26 of 26)

- [Disentangled Explanations of Neural Network Predictions by Finding Relevant Subspaces](https://mlsanity.com/paper/2212.14855/) (2022)
- [Tailoring Self-Rationalizers with Multi-Reward Distillation](https://mlsanity.com/paper/2311.02805/) (2023)
- [A Unified Game-Theoretic Interpretation of Adversarial Robustness](https://mlsanity.com/paper/2111.03536/) (2021)
- [TextGenSHAP: Scalable Post-hoc Explanations in Text Generation with Long Documents](https://mlsanity.com/paper/2312.01279/) (2023)
- [Interpreting Attributions and Interactions of Adversarial Attacks](https://mlsanity.com/paper/2108.06895/) (2021)
- [Multi-Level Explanations for Generative Language Models](https://mlsanity.com/paper/2403.14459/) (2024)
- [Can We Faithfully Represent Masked States to Compute Shapley Values on a DNN?](https://mlsanity.com/paper/2105.10719/) (2021)
- [A Comprehensive Review on Financial Explainable AI](https://mlsanity.com/paper/2309.11960/) (2023)
- [Exploring Distantly-Labeled Rationales in Neural Network Models](https://mlsanity.com/paper/2106.01809/) (2021)
- [Feature Interactions Reveal Linguistic Structure in Language Models](https://mlsanity.com/paper/2306.12181/) (2023)
- [ER-Test: Evaluating Explanation Regularization Methods for Language Models](https://mlsanity.com/paper/2205.12542/) (2022)
- [Interpretable Artificial Intelligence through the Lens of Feature Interaction](https://mlsanity.com/paper/2103.03103/) (2021)
- [Hierarchical Interpretation of Neural Text Classification](https://mlsanity.com/paper/2202.09792/) (2022)
- [Dynamically Refined Regularization for Improving Cross-corpora Hate Speech Detection](https://mlsanity.com/paper/2203.12536/) (2022)
- [Extracting Lexical Features from Dialects via Interpretable Dialect Classifiers](https://mlsanity.com/paper/2402.17914/) (2024)
- [CAVE: Controllable Authorship Verification Explanations](https://mlsanity.com/paper/2406.16672/) (2024)
- [Explainable Verbal Deception Detection using Transformers](https://mlsanity.com/paper/2210.03080/) (2022)
- [Advancing Interpretability in Text Classification through Prototype Learning](https://mlsanity.com/paper/2410.17546/) (2024)
- [AutoTriggER: Label-Efficient and Robust Named Entity Recognition with Auxiliary Trigger Extraction](https://mlsanity.com/paper/2109.04726/) (2021)
- [Unsupervised Keyphrase Extraction via Interpretable Neural Networks](https://mlsanity.com/paper/2203.07640/) (2022)
- [Transparent Neighborhood Approximation for Text Classifier Explanation](https://mlsanity.com/paper/2411.16251/) (2024)
- [A Unified Game-Theoretic Interpretation of Adversarial Robustness](https://mlsanity.com/paper/2103.07364/) (2021)
- [Domain Classification-based Source-specific Term Penalization for Domain Adaptation in Hate-speech Detection](https://mlsanity.com/paper/2209.08681/) (2022)
- [PE: A Poincare Explanation Method for Fast Text Hierarchy Generation](https://mlsanity.com/paper/2403.16554/) (2024)
- [Concept Based Continuous Prompts for Interpretable Text Classification](https://mlsanity.com/paper/2412.01644/) (2024)
- [PLEX: Perturbation-free Local Explanations for LLM-Based Text Classification](https://mlsanity.com/paper/2507.10596/) (2025)

## Abstract (arXiv)

The impressive performance of neural networks on natural language processing tasks attributes to their ability to model complicated word and phrase compositions. To explain how the model handles semantic compositions, we study hierarchical explanation of neural network predictions. We identify non-additivity and context independent importance attributions within hierarchies as two desirable properties for highlighting word and phrase compositions. We show some prior efforts on hierarchical explanations, e.g. contextual decomposition, do not satisfy the desired properties mathematically, leading to inconsistent explanation quality in different models. In this paper, we start by proposing a formal and general way to quantify the importance of each word and phrase. Following the formulation, we propose Sampling and Contextual Decomposition (SCD) algorithm and Sampling and Occlusion (SOC) algorithm. Human and metrics evaluation on both LSTM models and BERT Transformer models on multiple datasets show that our algorithms outperform prior hierarchical explanation algorithms. Our algorithms help to visualize semantic composition captured by models, extract classification rules and improve human trust of models. Project page: https://inklab.usc.edu/hiexpl/