# Pruning and Sparsemax Methods for Hierarchical Attention Networks

arXiv: https://arxiv.org/abs/2004.04343 · HTML: https://mlsanity.com/paper/2004.04343/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 8 Apr 2020.

## Built on by (most cited first, 1 of 1)

- [Towards Objective and Unbiased Decision Assessments with LLM-Enhanced Hierarchical Attention Networks](https://mlsanity.com/paper/2411.08504/) (2024)

## Abstract (arXiv)

This paper introduces and evaluates two novel Hierarchical Attention Network models [Yang et al., 2016] - i) Hierarchical Pruned Attention Networks, which remove the irrelevant words and sentences from the classification process in order to reduce potential noise in the document classification accuracy and ii) Hierarchical Sparsemax Attention Networks, which replace the Softmax function used in the attention mechanism with the Sparsemax [Martins and Astudillo, 2016], capable of better handling importance distributions where a lot of words or sentences have very low probabilities. Our empirical evaluation on the IMDB Review for sentiment analysis datasets shows both approaches to be able to match the results obtained by the current state-of-the-art (without, however, any significant benefits). All our source code is made available athttps://github.com/jmribeiro/dsl-project.