# Multilingual Hierarchical Attention Networks for Document Classification

arXiv: https://arxiv.org/abs/1707.00896 · HTML: https://mlsanity.com/paper/1707.00896/

## Key facts

- Cited by 9 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 4 Jul 2017.

## Built on by (most cited first, 9 of 9)

- [Weakly-supervised Text Classification Based on Keyword Graph](https://mlsanity.com/paper/2110.02591/) (2021)
- [Exploring Discourse Structures for Argument Impact Classification](https://mlsanity.com/paper/2106.00976/) (2021)
- [Attention-based Dynamic Multilayer Graph Neural Networks for Loan Default Prediction](https://mlsanity.com/paper/2402.00299/) (2024)
- [Emotion Prediction Oriented method with Multiple Supervisions for Emotion-Cause Pair Extraction](https://mlsanity.com/paper/2302.12417/) (2023)
- [A study on the impact of pre-trained model on Just-In-Time defect prediction](https://mlsanity.com/paper/2309.02317/) (2023)
- [Open Problems in Applied Deep Learning](https://mlsanity.com/paper/2301.11316/) (2023)
- [More Context, Larger Models, or Moral Knowledge? A Systematic Study of Schwartz Value Detection in Political Texts](https://mlsanity.com/paper/2605.22641/) (2026)
- [Sequential Attention Module for Natural Language Processing](https://mlsanity.com/paper/2109.03009/) (2021)
- [A General-Purpose Multilingual Document Encoder](https://mlsanity.com/paper/2305.07016/) (2023)

## Abstract (arXiv)

Hierarchical attention networks have recently achieved remarkable performance for document classification in a given language. However, when multilingual document collections are considered, training such models separately for each language entails linear parameter growth and lack of cross-language transfer. Learning a single multilingual model with fewer parameters is therefore a challenging but potentially beneficial objective. To this end, we propose multilingual hierarchical attention networks for learning document structures, with shared encoders and/or shared attention mechanisms across languages, using multi-task learning and an aligned semantic space as input. We evaluate the proposed models on multilingual document classification with disjoint label sets, on a large dataset which we provide, with 600k news documents in 8 languages, and 5k labels. The multilingual models outperform monolingual ones in low-resource as well as full-resource settings, and use fewer parameters, thus confirming their computational efficiency and the utility of cross-language transfer.