# Neural semi-Markov CRF for Monolingual Word Alignment

arXiv: https://arxiv.org/abs/2106.02569 · HTML: https://mlsanity.com/paper/2106.02569/

## Key facts

- Cited by 16 papers in the map (data through 14 Oct 2026).
- Builds on 25 of its 64 references that are in the map.
- First posted on arXiv 4 Jun 2021.

## Builds on (most cited first, 25 of 25)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [GLUE: A Multi-Task Benchmark and Analysis Platform for Natural Language Understanding](https://mlsanity.com/paper/1804.07461/) (2018)
- [Delving Deep into Rectifiers: Surpassing Human-Level Performance on ImageNet Classification](https://mlsanity.com/paper/1502.01852/) (2015)
- [A Broad-Coverage Challenge Corpus for Sentence Understanding through Inference](https://mlsanity.com/paper/1704.05426/) (2017)
- [HuggingFace's Transformers: State-of-the-art Natural Language Processing](https://mlsanity.com/paper/1910.03771/) (2019)
- [A large annotated corpus for learning natural language inference](https://mlsanity.com/paper/1508.05326/) (2015)
- [SemEval-2017 Task 1: Semantic Textual Similarity - Multilingual and Cross-lingual Focused Evaluation](https://mlsanity.com/paper/1708.00055/) (2017)
- [SpanBERT: Improving Pre-training by Representing and Predicting Spans](https://mlsanity.com/paper/1907.10529/) (2019)
- [AllenNLP: A Deep Semantic Natural Language Processing Platform](https://mlsanity.com/paper/1803.07640/) (2018)
- [Sentence Simplification with Deep Reinforcement Learning](https://mlsanity.com/paper/1703.10931/) (2017)
- [Neural CRF Model for Sentence Alignment in Text Simplification](https://mlsanity.com/paper/2005.02324/) (2020)
- [Automatically Neutralizing Subjective Bias in Text](https://mlsanity.com/paper/1911.09709/) (2019)
- [Jointly Learning to Align and Translate with Transformer Models](https://mlsanity.com/paper/1909.02074/) (2019)
- [A Continuously Growing Dataset of Sentential Paraphrases](https://mlsanity.com/paper/1708.00391/) (2017)
- [EditNTS: An Neural Programmer-Interpreter Model for Sentence Simplification through Explicit Editing](https://mlsanity.com/paper/1906.08104/) (2019)
- [Controllable Text Simplification with Explicit Paraphrasing](https://mlsanity.com/paper/2010.11004/) (2020)
- [Learning To Split and Rephrase From Wikipedia Edit History](https://mlsanity.com/paper/1808.09468/) (2018)
- [End-to-End Neural Word Alignment Outperforms GIZA++](https://mlsanity.com/paper/2004.14675/) (2020)
- [ParaSCI: A Large Scientific Paraphrase Dataset for Longer Paraphrase Generation](https://mlsanity.com/paper/2101.08382/) (2021)
- [A Supervised Word Alignment Method based on Cross-Language Span Prediction using Multilingual BERT](https://mlsanity.com/paper/2004.14516/) (2020)
- [Neural Network Models for Paraphrase Identification, Semantic Textual Similarity, Natural Language Inference, and Question Answering](https://mlsanity.com/paper/1806.04330/) (2018)
- [Discourse Level Factors for Sentence Deletion in Text Simplification](https://mlsanity.com/paper/1911.10384/) (2019)
- [A Corpus of Sentence-level Revisions in Academic Writing: A Step towards Understanding Statement Strength in Communication](https://mlsanity.com/paper/1405.1439/) (2014)
- [A Discriminative Neural Model for Cross-Lingual Word Alignment](https://mlsanity.com/paper/1909.00444/) (2019)
- [Iterative Paraphrastic Augmentation with Discriminative Span Alignment](https://mlsanity.com/paper/2007.00320/) (2020)

## Built on by (most cited first, 16 of 16)

- [LENS: A Learnable Evaluation Metric for Text Simplification](https://mlsanity.com/paper/2212.09739/) (2022)
- [Frustratingly Easy Label Projection for Cross-lingual Transfer](https://mlsanity.com/paper/2211.15613/) (2022)
- [arXivEdits: Understanding the Human Revision Process in Scientific Writing](https://mlsanity.com/paper/2210.15067/) (2022)
- [Towards Explainable Evaluation Metrics for Natural Language Generation](https://mlsanity.com/paper/2203.11131/) (2022)
- [Translation and Fusion Improves Zero-shot Cross-lingual Information Extraction](https://mlsanity.com/paper/2305.13582/) (2023)
- [Dancing Between Success and Failure: Edit-level Simplification Evaluation using SALSA](https://mlsanity.com/paper/2305.14458/) (2023)
- [Multilingual Simplification of Medical Texts](https://mlsanity.com/paper/2305.12532/) (2023)
- [Explaining with Contrastive Phrasal Highlighting: A Case Study in Assisting Humans to Detect Translation Differences](https://mlsanity.com/paper/2312.01582/) (2023)
- [Unbalanced Optimal Transport for Unbalanced Word Alignment](https://mlsanity.com/paper/2306.04116/) (2023)
- [Thresh: A Unified, Customizable and Deployable Platform for Fine-Grained Text Evaluation](https://mlsanity.com/paper/2308.06953/) (2023)
- [Word Alignment in the Era of Deep Learning: A Tutorial](https://mlsanity.com/paper/2212.00138/) (2022)
- [OTTAWA: Optimal TransporT Adaptive Word Aligner for Hallucination and Omission Translation Errors Detection](https://mlsanity.com/paper/2406.01919/) (2024)
- [Edit-Constrained Decoding for Sentence Simplification](https://mlsanity.com/paper/2409.19247/) (2024)
- [DM-Align: Leveraging the Power of Natural Language Instructions to Make Changes to Images](https://mlsanity.com/paper/2404.18020/) (2024)
- [Hallucination Span Detection with Input-Side Evidence Alignment](https://mlsanity.com/paper/2608.15804/) (2026)
- [Aligning Faithfulness for Theory-Scale Autoformalization](https://mlsanity.com/paper/2610.18498/) (2026)

## Abstract (arXiv)

Monolingual word alignment is important for studying fine-grained editing operations (i.e., deletion, addition, and substitution) in text-to-text generation tasks, such as paraphrase generation, text simplification, neutralizing biased language, etc. In this paper, we present a novel neural semi-Markov CRF alignment model, which unifies word and phrase alignments through variable-length spans. We also create a new benchmark with human annotations that cover four different text genres to evaluate monolingual word alignment models in more realistic settings. Experimental results show that our proposed model outperforms all previous approaches for monolingual word alignment as well as a competitive QA-based baseline, which was previously only applied to bilingual data. Our model demonstrates good generalizability to three out-of-domain datasets and shows great utility in two downstream applications: automatic text simplification and sentence pair classification tasks.