# Exact Hard Monotonic Attention for Character-Level Transduction

arXiv: https://arxiv.org/abs/1905.06319 · HTML: https://mlsanity.com/paper/1905.06319/

## Key facts

- Cited by 22 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 15 May 2019.

## Built on by (most cited first, 22 of 22)

- [Can a Transformer Pass the Wug Test? Tuning Copying Bias in Neural Morphological Inflection Models](https://mlsanity.com/paper/2104.06483/) (2021)
- [Inducing and Using Alignments for Transition-based AMR Parsing](https://mlsanity.com/paper/2205.01464/) (2022)
- [Benchmarking Compositionality with Formal Languages](https://mlsanity.com/paper/2208.08195/) (2022)
- [Towards a Broad Coverage Named Entity Resource: A Data-Efficient Approach for Many Diverse Languages](https://mlsanity.com/paper/2201.12219/) (2022)
- [How Suitable Are Subword Segmentation Strategies for Translating Non-Concatenative Morphology?](https://mlsanity.com/paper/2109.01100/) (2021)
- [SIP: Injecting a Structural Inductive Bias into a Seq2Seq Model by Simulation](https://mlsanity.com/paper/2310.00796/) (2023)
- [Compositional Generalisation with Structured Reordering and Fertility Layers](https://mlsanity.com/paper/2210.03183/) (2022)
- [On Biasing Transformer Attention Towards Monotonicity](https://mlsanity.com/paper/2104.03945/) (2021)
- [Frequency matters: Modeling irregular morphological patterns in Spanish with Transformers](https://mlsanity.com/paper/2410.21013/) (2024)
- [Text normalization for low-resource languages: the case of Ligurian](https://mlsanity.com/paper/2206.07861/) (2022)
- [On the Role of Morphological Information for Contextual Lemmatization](https://mlsanity.com/paper/2302.00407/) (2023)
- [Searching for Search Errors in Neural Morphological Inflection](https://mlsanity.com/paper/2102.08424/) (2021)
- [Computational Morphology with Neural Network Approaches](https://mlsanity.com/paper/2105.09404/) (2021)
- [On the Learning Dynamics of Attention Networks](https://mlsanity.com/paper/2307.13421/) (2023)
- [Comparative Error Analysis in Neural and Finite-state Models for Unsupervised Character-level Transduction](https://mlsanity.com/paper/2106.12698/) (2021)
- [Searching for More Efficient Dynamic Programs](https://mlsanity.com/paper/2109.06966/) (2021)
- [Transformers on Multilingual Clause-Level Morphology](https://mlsanity.com/paper/2211.01736/) (2022)
- [Mind Your Moras: Orthography-Aware Error Analysis of Neural Japanese Morphological Generation](https://mlsanity.com/paper/2605.20043/) (2026)
- [When Irregularity Helps: A Subclass Analysis of Inductive Bias in Neural Morphology](https://mlsanity.com/paper/2605.20558/) (2026)
- [Differentiable Generative Phonology](https://mlsanity.com/paper/2102.05717/) (2021)
- [A Framework for Bidirectional Decoding: Case Study in Morphological Inflection](https://mlsanity.com/paper/2305.12580/) (2023)
- [Improving Low-Resource Morphological Inflection via Self-Supervised Objectives](https://mlsanity.com/paper/2506.05227/) (2025)

## Abstract (arXiv)

Many common character-level, string-to string transduction tasks, e.g., grapheme-tophoneme conversion and morphological inflection, consist almost exclusively of monotonic transductions. However, neural sequence-to sequence models that use non-monotonic soft attention often outperform popular monotonic models. In this work, we ask the following question: Is monotonicity really a helpful inductive bias for these tasks? We develop a hard attention sequence-to-sequence model that enforces strict monotonicity and learns a latent alignment jointly while learning to transduce. With the help of dynamic programming, we are able to compute the exact marginalization over all monotonic alignments. Our models achieve state-of-the-art performance on morphological inflection. Furthermore, we find strong performance on two other character-level transduction tasks. Code is available at https://github.com/shijie-wu/neural-transducer.