# Multilevel Text Alignment with Cross-Document Attention

arXiv: https://arxiv.org/abs/2010.01263 · HTML: https://mlsanity.com/paper/2010.01263/

## Key facts

- Cited by 7 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 3 Oct 2020.

## Built on by (most cited first, 7 of 7)

- [Do Language Models Plagiarize?](https://mlsanity.com/paper/2203.07618/) (2022)
- [Multi-Vector Models with Textual Guidance for Fine-Grained Scientific Document Similarity](https://mlsanity.com/paper/2111.08366/) (2021)
- [CDLM: Cross-Document Language Modeling](https://mlsanity.com/paper/2101.00406/) (2021)
- [Self-Supervised Document Similarity Ranking via Contextualized Language Models and Hierarchical Inference](https://mlsanity.com/paper/2106.01186/) (2021)
- [Sequential Cross-Document Coreference Resolution](https://mlsanity.com/paper/2104.08413/) (2021)
- [Match-Ignition: Plugging PageRank into Transformer for Long-form Text Matching](https://mlsanity.com/paper/2101.06423/) (2021)
- [An Interactive UI to Support Sensemaking over Collections of Parallel Texts](https://mlsanity.com/paper/2303.06264/) (2023)

## Abstract (arXiv)

Text alignment finds application in tasks such as citation recommendation and plagiarism detection. Existing alignment methods operate at a single, predefined level and cannot learn to align texts at, for example, sentence and document levels. We propose a new learning approach that equips previously established hierarchical attention encoders for representing documents with a cross-document attention component, enabling structural comparisons across different levels (document-to-document and sentence-to-document). Our component is weakly supervised from document pairs and can align at multiple levels. Our evaluation on predicting document-to-document relationships and sentence-to-document relationships on the tasks of citation recommendation and plagiarism detection shows that our approach outperforms previously established hierarchical, attention encoders based on recurrent and transformer contextualization that are unaware of structural correspondence between documents.