# Unbalanced Optimal Transport for Unbalanced Word Alignment

arXiv: https://arxiv.org/abs/2306.04116 · HTML: https://mlsanity.com/paper/2306.04116/

## Key facts

- Cited by 6 papers in the map (data through 14 Oct 2026).
- Builds on 37 of its 84 references that are in the map.
- First posted on arXiv 7 Jun 2023.

## Builds on (most cited first, 30 of 37)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [BERTScore: Evaluating Text Generation with BERT](https://mlsanity.com/paper/1904.09675/) (2019)
- [SQuAD: 100,000+ Questions for Machine Comprehension of Text](https://mlsanity.com/paper/1606.05250/) (2016)
- [Sinkhorn Distances: Lightspeed Computation of Optimal Transportation Distances](https://mlsanity.com/paper/1306.0895/) (2013)
- [Adafactor: Adaptive Learning Rates with Sublinear Memory Cost](https://mlsanity.com/paper/1804.04235/) (2018)
- [SpanBERT: Improving Pre-training by Representing and Predicting Spans](https://mlsanity.com/paper/1907.10529/) (2019)
- [How Contextual are Contextualized Word Representations? Comparing the Geometry of BERT, ELMo, and GPT-2 Embeddings](https://mlsanity.com/paper/1909.00512/) (2019)
- [MoverScore: Text Generation Evaluating with Contextualized Embeddings and Earth Mover Distance](https://mlsanity.com/paper/1909.02622/) (2019)
- [Learning with a Wasserstein Loss](https://mlsanity.com/paper/1506.05439/) (2015)
- [Word Alignment by Fine-tuning Embeddings on Parallel Corpora](https://mlsanity.com/paper/2101.08231/) (2021)
- [SimAlign: High Quality Word Alignments without Parallel Training Data using Static and Contextualized Embeddings](https://mlsanity.com/paper/2004.08728/) (2020)
- [Neural CRF Model for Sentence Alignment in Text Simplification](https://mlsanity.com/paper/2005.02324/) (2020)
- [Unsupervised Alignment of Embeddings with Wasserstein Procrustes](https://mlsanity.com/paper/1805.11222/) (2018)
- [Jointly Learning to Align and Translate with Transformer Models](https://mlsanity.com/paper/1909.02074/) (2019)
- [Assessing Phrasal Representation and Composition in Transformers](https://mlsanity.com/paper/2010.03763/) (2020)
- [Scaling Algorithms for Unbalanced Transport Problems](https://mlsanity.com/paper/1607.05816/) (2016)
- [Multi-Vector Models with Textual Guidance for Fine-Grained Scientific Document Similarity](https://mlsanity.com/paper/2111.08366/) (2021)
- [Multi-Hop Paragraph Retrieval for Open-Domain Question Answering](https://mlsanity.com/paper/1906.06606/) (2019)
- [Improving Sequence-to-Sequence Learning via Optimal Transport](https://mlsanity.com/paper/1901.06283/) (2019)
- [Word Rotator's Distance](https://mlsanity.com/paper/2004.15003/) (2020)
- [Structured Optimal Transport](https://mlsanity.com/paper/1712.06199/) (2017)
- [Unbalanced Optimal Transport through Non-negative Penalized Linear Regression](https://mlsanity.com/paper/2106.04145/) (2021)
- [End-to-End Neural Word Alignment Outperforms GIZA++](https://mlsanity.com/paper/2004.14675/) (2020)
- [A Supervised Word Alignment Method based on Cross-Language Span Prediction using Multilingual BERT](https://mlsanity.com/paper/2004.14516/) (2020)
- [Rationalizing Text Matching: Learning Sparse Alignments via Optimal Transport](https://mlsanity.com/paper/2005.13111/) (2020)
- [Using Optimal Transport as Alignment Objective for fine-tuning Multilingual Contextualized Embeddings](https://mlsanity.com/paper/2110.02887/) (2021)
- [Neural semi-Markov CRF for Monolingual Word Alignment](https://mlsanity.com/paper/2106.02569/) (2021)
- [QA-Align: Representing Cross-Text Content Overlap by Aligning Question-Answer Propositions](https://mlsanity.com/paper/2109.12655/) (2021)
- [Toward Interpretable Semantic Textual Similarity via Optimal Transport-based Contrastive Sentence Learning](https://mlsanity.com/paper/2202.13196/) (2022)
- [Improving Text Generation with Student-Forcing Optimal Transport](https://mlsanity.com/paper/2010.05994/) (2020)

## Built on by (most cited first, 6 of 6)

- [Submodular Framework for Structured-Sparse Optimal Transport](https://mlsanity.com/paper/2406.04914/) (2024)
- [Policy-based Sentence Simplification: Replacing Parallel Corpora with LLM-as-a-Judge](https://mlsanity.com/paper/2512.06228/) (2025)
- [Distributional Open-Ended Evaluation of LLM Cultural Value Alignment Based on Value Codebook](https://mlsanity.com/paper/2604.06210/) (2026)
- [Grammatical Error Correction Evaluation by Optimally Transporting Edit Representation](https://mlsanity.com/paper/2602.05419/) (2026)
- [Hallucination Span Detection with Input-Side Evidence Alignment](https://mlsanity.com/paper/2608.15804/) (2026)
- [Optimal transport meets speech: a tutorial review](https://mlsanity.com/paper/2609.31787/) (2026)

## Abstract (arXiv)

Monolingual word alignment is crucial to model semantic interactions between sentences. In particular, null alignment, a phenomenon in which words have no corresponding counterparts, is pervasive and critical in handling semantically divergent sentences. Identification of null alignment is useful on its own to reason about the semantic similarity of sentences by indicating there exists information inequality. To achieve unbalanced word alignment that values both alignment and null alignment, this study shows that the family of optimal transport (OT), i.e., balanced, partial, and unbalanced OT, are natural and powerful approaches even without tailor-made techniques. Our extensive experiments covering unsupervised and supervised settings indicate that our generic OT-based alignment methods are competitive against the state-of-the-arts specially designed for word alignment, remarkably on challenging datasets with high null alignment frequencies.