# Hierarchical Document Encoder for Parallel Corpus Mining

arXiv: https://arxiv.org/abs/1906.08401 · HTML: https://mlsanity.com/paper/1906.08401/

## Key facts

- Cited by 10 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 20 Jun 2019.

## Built on by (most cited first, 10 of 10)

- [LongT5: Efficient Text-To-Text Transformer for Long Sequences](https://mlsanity.com/paper/2112.07916/) (2021)
- [Embarrassingly Easy Document-Level MT Metrics: How to Convert Any Pretrained Metric Into a Document-Level Metric](https://mlsanity.com/paper/2209.13654/) (2022)
- [CDLM: Cross-Document Language Modeling](https://mlsanity.com/paper/2101.00406/) (2021)
- [Generating Coherent Narratives by Learning Dynamic and Discrete Entity States with a Contrastive Framework](https://mlsanity.com/paper/2208.03985/) (2022)
- [LAWDR: Language-Agnostic Weighted Document Representations from Pre-trained Models](https://mlsanity.com/paper/2106.03379/) (2021)
- [Exploiting Parallel Corpora to Improve Multilingual Embedding based Document and Sentence Alignment](https://mlsanity.com/paper/2106.06766/) (2021)
- [Metric Learning in Multilingual Sentence Similarity Measurement for Document Alignment](https://mlsanity.com/paper/2108.09495/) (2021)
- [Pralekha: Cross-Lingual Document Alignment for Indic Languages](https://mlsanity.com/paper/2411.19096/) (2024)
- [A General-Purpose Multilingual Document Encoder](https://mlsanity.com/paper/2305.07016/) (2023)
- [Smart Bilingual Focused Crawling of Parallel Documents](https://mlsanity.com/paper/2405.14779/) (2024)

## Abstract (arXiv)

We explore using multilingual document embeddings for nearest neighbor mining of parallel data. Three document-level representations are investigated: (i) document embeddings generated by simply averaging multilingual sentence embeddings; (ii) a neural bag-of-words (BoW) document encoding model; (iii) a hierarchical multilingual document encoder (HiDE) that builds on our sentence-level model. The results show document embeddings derived from sentence-level averaging are surprisingly effective for clean datasets, but suggest models trained hierarchically at the document-level are more effective on noisy data. Analysis experiments demonstrate our hierarchical models are very robust to variations in the underlying sentence embedding quality. Using document embeddings trained with HiDE achieves state-of-the-art performance on United Nations (UN) parallel document mining, 94.9% P@1 for en-fr and 97.3% P@1 for en-es.