# Contrastive Unsupervised Word Alignment with Non-Local Features

arXiv: https://arxiv.org/abs/1410.2082 · HTML: https://mlsanity.com/paper/1410.2082/

## Key facts

- Cited by 11 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 8 Oct 2014.

## Built on by (most cited first, 11 of 11)

- [Word Alignment by Fine-tuning Embeddings on Parallel Corpora](https://mlsanity.com/paper/2101.08231/) (2021)
- [Weakly Supervised Contrastive Learning for Chest X-Ray Report Generation](https://mlsanity.com/paper/2109.12242/) (2021)
- [Finding Order in Chaos: A Novel Data Augmentation Method for Time Series in Contrastive Learning](https://mlsanity.com/paper/2309.13439/) (2023)
- [Towards User-Driven Neural Machine Translation](https://mlsanity.com/paper/2106.06200/) (2021)
- [Multilingual Sentence Transformer as A Multilingual Word Aligner](https://mlsanity.com/paper/2301.12140/) (2023)
- [Machine-Created Universal Language for Cross-lingual Transfer](https://mlsanity.com/paper/2305.13071/) (2023)
- [C3: Compositional Counterfactual Contrastive Learning for Video-grounded Dialogues](https://mlsanity.com/paper/2106.08914/) (2021)
- [FILM: How can Few-Shot Image Classification Benefit from Pre-Trained Language Models?](https://mlsanity.com/paper/2307.04114/) (2023)
- [StatioCL: Contrastive Learning for Time Series via Non-Stationary and Temporal Contrast](https://mlsanity.com/paper/2410.10048/) (2024)
- [Scaling Unsupervised Word Alignment to Documents via Structural Constraints](https://mlsanity.com/paper/2608.21023/) (2026)
- [TransAlign: Machine Translation Encoders are Strong Word Aligners, Too](https://mlsanity.com/paper/2510.27337/) (2025)

## Abstract (arXiv)

Word alignment is an important natural language processing task that indicates the correspondence between natural languages. Recently, unsupervised learning of log-linear models for word alignment has received considerable attention as it combines the merits of generative and discriminative approaches. However, a major challenge still remains: it is intractable to calculate the expectations of non-local features that are critical for capturing the divergence between natural languages. We propose a contrastive approach that aims to differentiate observed training examples from noises. It not only introduces prior knowledge to guide unsupervised learning but also cancels out partition functions. Based on the observation that the probability mass of log-linear models for word alignment is usually highly concentrated, we propose to use top-n alignments to approximate the expectations with respect to posterior distributions. This allows for efficient and accurate calculation of expectations of non-local features. Experiments show that our approach achieves significant improvements over state-of-the-art unsupervised word alignment methods.