mlsanityData to 14 Oct 2026

How Suitable Are Subword Segmentation Strategies for Translating Non-Concatenative Morphology?

arXiv 2109.01100 ↗First posted 2 Sep 2021cs.CLcited by 7

Not on the map. Why?

Abstract (arXiv)

Data-driven subword segmentation has become the default strategy for open-vocabulary machine translation and other NLP tasks, but may not be sufficiently generic for optimal learning of non-concatenative morphology. We design a test suite to evaluate segmentation strategies on different types of morphological phenomena in a controlled, semi-synthetic setting. In our experiments, we compare how well machine translation models trained on subword- and character-level can translate these morphological phenomena. We find that learning to analyse and generate morphologically complex surface representations is still challenging, especially for non-concatenative morphological phenomena like reduplication or vowel harmony and for rare word stems. Based on our results, we recommend that novel text representation strategies be tested on a range of typologically diverse languages to minimise the risk of adopting a strategy that inadvertently disadvantages certain languages.

Key facts. Cited by 7 papers in the map (data through 14 Oct 2026). Builds on 33 of its 59 references that are in the map. First posted on arXiv 2 Sep 2021.

Builds on

33 of 59 references.

References · 33

  1. 2017Attention Is All You Needcited by 60,361
  2. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,399
  3. 2014Adam: A Method for Stochastic Optimizationcited by 37,240
  4. 2020Language Models are Few-Shot Learnerscited by 32,419
  5. 2015Neural Machine Translation of Rare Words with Subword Unitscited by 2,918
  6. 2018SentencePiece: A simple and language independent subword tokenizer and detokenizer for Neural Text Processingcited by 1,989
  7. 2018A Call for Clarity in Reporting BLEU Scorescited by 1,799
  8. 2018Subword Regularization: Improving Neural Network Translation Models with Multiple Subword Candidatescited by 639
  9. 2019BLiMP: The Benchmark of Linguistic Minimal Pairs for Englishcited by 564
  10. 2017Synthetic and Natural Noise Both Break Neural Machine Translationcited by 306
  11. 2016Using the Output Embedding to Improve Language Modelscited by 272
  12. 2018Targeted Syntactic Evaluation of Language Modelscited by 242
21 more
  1. 2021CANINE: Pre-training an Efficient Tokenization-Free Encoder for Language Representationcited by 232
  2. 2019BPE-Dropout: Simple and Effective Subword Regularizationcited by 195
  3. 2017What do Neural Machine Translation Models Learn about Morphology?cited by 166
  4. 2019Combating Adversarial Misspellings with Robust Word Recognitioncited by 163
  5. 2020CharacterBERT: Reconciling ELMo and BERT for Word-Level Open-Vocabulary Representations From Characterscited by 87
  6. 2020Improving Multilingual Models with Language-Clustered Vocabulariescited by 53
  7. 2021Robust Open-Vocabulary Translation from Visual Text Representationscited by 47
  8. 2021Multi-view Subword Regularizationcited by 40
  9. 2016How Grammatical is Character-level Neural Machine Translation? Assessing MT Quality with Contrastive Translation Pairscited by 39
  10. 2020SIGMORPHON 2020 Shared Task 0: Typologically Diverse Morphological Inflectioncited by 39
  11. 2019On the Linguistic Representational Power of Neural Machine Translation Modelscited by 34
  12. 2020Scheduled DropHead: A Regularization Method for Transformer Modelscited by 28
  13. 2019Exact Hard Monotonic Attention for Character-Level Transductioncited by 22
  14. 2017The Galactic Dependencies Treebanks: Getting More Data by Synthesizing New Languagescited by 19
  15. 2017Nematus: a Toolkit for Neural Machine Translationcited by 16
  16. 2017From Characters to Words to in Between: Do We Capture Morphology?cited by 13
  17. 2018Compositional Representation of Morphologically-Rich Input for Neural Machine Translationcited by 12
  18. 2016Word Representation Models for Morphologically Rich Languages in Neural Machine Translationcited by 9
  19. 2017Modeling Target-Side Inflection in Neural Machine Translationcited by 9
  20. 2020Multi-Adversarial Learning for Cross-Lingual Word Embeddingscited by 7
  21. 2021On Biasing Transformer Attention Towards Monotonicitycited by 5

Built on by

Cited by year

Citing papers per year: 2021: 1; 2022: 4; 2023: 1; 2024: 0; 2025: 1.

7 papers.

  1. 2021Between words and characters: A Brief History of Open-Vocabulary Modeling and Tokenization in NLPcited by 132
  2. 2022Language Modelling with Pixelscited by 55
  3. 2022How can NLP Help Revitalize Endangered Languages? A Case Study and Roadmap for the Cherokee Languagecited by 19
  4. 2025Confounding Factors in Relating Model Performance to Morphologycited by 14
  5. 2022DivEMT: Neural Machine Translation Post-Editing Effort Across Typologically Diverse Languagescited by 11
  6. 2022Modeling Target-Side Morphology in Neural Machine Translation: A Comparison of Strategiescited by 1
  7. 2023Impact of Subword Pooling Strategy on Cross-lingual Event Detectioncited by 0