# Scaling Unsupervised Word Alignment to Documents via Structural Constraints

arXiv: https://arxiv.org/abs/2608.21023 · HTML: https://mlsanity.com/paper/2608.21023/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 29 of its 49 references that are in the map.
- First posted on arXiv 21 Aug 2026.

## Builds on (most cited first, 29 of 29)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [BERTScore: Evaluating Text Generation with BERT](https://mlsanity.com/paper/1904.09675/) (2019)
- [Unsupervised Cross-lingual Representation Learning at Scale](https://mlsanity.com/paper/1911.02116/) (2019)
- [Qwen3 Embedding: Advancing Text Embedding and Reranking Through Foundation Models](https://mlsanity.com/paper/2506.05176/) (2025)
- [Language-agnostic BERT Sentence Embedding](https://mlsanity.com/paper/2007.01852/) (2020)
- [Experts, Errors, and Context: A Large-Scale Study of Human Evaluation for Machine Translation](https://mlsanity.com/paper/2104.14478/) (2021)
- [Word Alignment by Fine-tuning Embeddings on Parallel Corpora](https://mlsanity.com/paper/2101.08231/) (2021)
- [SimAlign: High Quality Word Alignments without Parallel Training Data using Static and Contextualized Embeddings](https://mlsanity.com/paper/2004.08728/) (2020)
- [Detecting Hallucinated Content in Conditional Neural Sequence Generation](https://mlsanity.com/paper/2011.02593/) (2020)
- [Coarse-to-Fine Decoding for Neural Semantic Parsing](https://mlsanity.com/paper/1805.04793/) (2018)
- [Detecting and Mitigating Hallucinations in Machine Translation: Model Internal Workings Alone Do Well, Sentence Similarity Even Better](https://mlsanity.com/paper/2212.08597/) (2022)
- [mmBERT: A Modern Multilingual Encoder with Annealed Language Learning](https://mlsanity.com/paper/2509.06888/) (2025)
- [HalOmi: A Manually Annotated Benchmark for Multilingual Hallucination and Omission Detection in Machine Translation](https://mlsanity.com/paper/2305.11746/) (2023)
- [TransLLaMa: LLM-based Simultaneous Translation System](https://mlsanity.com/paper/2402.04636/) (2024)
- [Frustratingly Easy Label Projection for Cross-lingual Transfer](https://mlsanity.com/paper/2211.15613/) (2022)
- [EuroBERT: Scaling Multilingual Encoders for European Languages](https://mlsanity.com/paper/2503.05500/) (2025)
- [Model and Data Transfer for Cross-Lingual Sequence Labelling in Zero-Resource Settings](https://mlsanity.com/paper/2210.12623/) (2022)
- [SemEval-2023 Task 2: Fine-grained Multilingual Named Entity Recognition (MultiCoNER 2)](https://mlsanity.com/paper/2305.06586/) (2023)
- [Word Alignment as Preference for Machine Translation](https://mlsanity.com/paper/2405.09223/) (2024)
- [Enhancing Cross-lingual Sentence Embedding for Low-resource Languages with Word Alignment](https://mlsanity.com/paper/2404.02490/) (2024)
- [Contrastive Unsupervised Word Alignment with Non-Local Features](https://mlsanity.com/paper/1410.2082/) (2014)
- [WSPAlign: Word Alignment Pre-training via Large-Scale Weakly Supervised Span Prediction](https://mlsanity.com/paper/2306.05644/) (2023)
- [Neural Machine Translation with Adequacy-Oriented Learning](https://mlsanity.com/paper/1811.08541/) (2018)
- [Multilingual Sentence Transformer as A Multilingual Word Aligner](https://mlsanity.com/paper/2301.12140/) (2023)
- [Towards Unsupervised Recognition of Token-level Semantic Differences in Related Documents](https://mlsanity.com/paper/2305.13303/) (2023)
- [BinaryAlign: Word Alignment as Binary Sequence Labeling](https://mlsanity.com/paper/2407.12881/) (2024)
- [SwissGov-RSD: A Human-annotated, Cross-lingual Benchmark for Token-level Recognition of Semantic Differences Between Related Documents](https://mlsanity.com/paper/2512.07538/) (2025)
- [OTTAWA: Optimal TransporT Adaptive Word Aligner for Hallucination and Omission Translation Errors Detection](https://mlsanity.com/paper/2406.01919/) (2024)
- [Explainable Semantic Textual Similarity via Dissimilar Span Detection](https://mlsanity.com/paper/2603.21174/) (2026)

## Built on by (most cited first, 1 of 1)

- [When Do Agents Help? Embedding, LLM and Agentic Alignment of Classical Texts and Their Translations](https://mlsanity.com/paper/2609.33691/) (2026)

## Abstract (arXiv)

Word alignment has traditionally been studied between sentences, but many cross-lingual tasks increasingly require correspondences across full documents. While recent multilingual embedding models can encode long inputs, we show that applying algorithms designed for sentences directly to documents leads to performance degradation. To address this, we introduce CTFAlign, a lightweight, training-free approach for document-level word alignment. CTFAlign applies a coarse-to-fine refinement strategy that restricts the alignment search space to semantically similar regions. Additionally, we introduce MDPAlign, a simpler alternative that constrains alignments by position with a main diagonal prior. Both approaches operate directly on full documents without relying on sentence segmentation or sentence alignment. We evaluate these methods across six language pairs varying in typological distance, resourcedness, and document length. Averaged over three models, CTFAlign reduces word alignment error rate from 0.412 to 0.326. These gains transfer downstream, leading to improvements in document-level translation coverage evaluation and recognition of semantic differences. We release CTFAlign as a Python package and make the code and data to reproduce our experiments publicly available.