mlsanityData to 7 Oct 2026

Why don't people use character-level machine translation?

arXiv 2110.08191 ↗First posted 15 Oct 2021cs.CLcited by 27

Not on the map. Why?

Abstract (arXiv)

We present a literature and empirical survey that critically assesses the state of the art in character-level modeling for machine translation (MT). Despite evidence in the literature that character-level systems are comparable with subword systems, they are virtually never used in competitive setups in WMT competitions. We empirically show that even with recent modeling innovations in character-level natural language processing, character-level MT systems still struggle to match their subword-based counterparts. Character-level MT systems show neither better domain robustness, nor better morphological generalization, despite being often so motivated. However, we are able to show robustness towards source side noise and that translation quality does not degrade with increasing beam size at decoding time.

Key facts. Cited by 27 papers in the map (data through 7 Oct 2026). Builds on 46 of its 74 references that are in the map. First posted on arXiv 15 Oct 2021.

Builds on

46 of 74 references.

References · 46

  1. 2017Attention Is All You Needcited by 59,932
  2. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  3. 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,491
  4. 2014Neural Machine Translation by Jointly Learning to Align and Translatecited by 4,530
  5. 2014Sequence to Sequence Learning with Neural Networkscited by 2,974
  6. 2015Neural Machine Translation of Rare Words with Subword Unitscited by 2,898
  7. 2018SentencePiece: A simple and language independent subword tokenizer and detokenizer for Neural Text Processingcited by 1,973
  8. 2020mT5: A massively multilingual pre-trained text-to-text transformercited by 1,813
  9. 2018A Call for Clarity in Reporting BLEU Scorescited by 1,790
  10. 2019fairseq: A Fast, Extensible Toolkit for Sequence Modelingcited by 1,486
  11. 2020COMET: A Neural Framework for MT Evaluationcited by 1,024
  12. 2021ByT5: Towards a token-free future with pre-trained byte-to-byte modelscited by 504
34 more
  1. 2015Highway Networkscited by 257
  2. 2019Evaluating Gender Bias in Machine Translationcited by 247
  3. 2021CANINE: Pre-training an Efficient Tokenization-Free Encoder for Language Representationcited by 229
  4. 2019BPE-Dropout: Simple and Effective Subword Regularizationcited by 189
  5. 2021Charformer: Fast Character Transformers via Gradient-based Subword Tokenizationcited by 158
  6. 2020Is MAP Decoding All You Need? The Inadequacy of the Mode in Neural Machine Translationcited by 137
  7. 2019On NMT Search Errors and Model Errors: Cat Got Your Tongue?cited by 114
  8. 2020If beam search is the answer, what was the question?cited by 114
  9. 2018Training Tips for the Transformer Modelcited by 85
  10. 2019Tagged Back-Translationcited by 76
  11. 2019Revisiting Low-Resource Neural Machine Translation: A Case Studycited by 61
  12. 2021Understanding the Properties of Minimum Bayes Risk Decoding in Neural Machine Translationcited by 56
  13. 2016Fully Character-Level Neural Machine Translation without Explicit Segmentationcited by 55
  14. 2016How Grammatical is Character-level Neural Machine Translation? Assessing MT Quality with Contrastive Translation Pairscited by 39
  15. 2018Training Deeper Neural Machine Translation Models with Transparent Attentioncited by 37
  16. 2018Revisiting Character-Based Neural Machine Translation with Capacity and Compressioncited by 28
  17. 2019How Decoding Strategies Affect the Verifiability of Generated Textcited by 27
  18. 2016A Character-Level Decoder without Explicit Segmentation for Neural Machine Translationcited by 21
  19. 2020Gender Coreference and Bias Evaluation at WMT 2020cited by 21
  20. 2016Achieving Open Vocabulary Neural Machine Translation with Hybrid Word-Character Modelscited by 18
  21. 2019Findings of the First Shared Task on Machine Translation Robustnesscited by 17
  22. 2020Announcing CzEng 2.0 Parallel Corpus with over 2 Gigawordscited by 17
  23. 2020Character-Level Translation with Self-attentioncited by 12
  24. 2018Learning to Segment Inputs for NMT Favors Character-Level Processingcited by 11
  25. 2019Character-based NMT with Transformercited by 10
  26. 2020Neural Machine Translation without Embeddingscited by 10
  27. 2020Character-level Transformer-based Neural Machine Translationcited by 7
  28. 2021What Do You Get When You Cross Beam Search with Nucleus Sampling?cited by 7
  29. 2020Why Neural Machine Translation Prefers Empty Outputscited by 6
  30. 2019On the Importance of Word Boundaries in Character-level Neural Machine Translationcited by 5
  31. 2018Character-level Chinese-English Translation through ASCII Encodingcited by 4
  32. 2018Neural Machine Translation of Logographic Languages Using Sub-character Level Informationcited by 4
  33. 2017The Helsinki Neural Machine Translation Systemcited by 2
  34. 2019JUCBNMT at WMT2018 News Translation Task: Character Based Neural Machine Translation of Finnish to Englishcited by 1

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 1; 2022: 6; 2023: 8; 2024: 5; 2025: 4; 2026: 3.

27 papers.

Sort:

  1. 2021Between words and characters: A Brief History of Open-Vocabulary Modeling and Tokenization in NLPcited by 130
  2. 2022State-of-the-art generalisation research in NLP: A taxonomy and reviewcited by 97
  3. 2022What do tokens know about their characters and how do they know it?cited by 36
  4. 2024Zero-Shot Tokenizer Transfercited by 30
  5. 2024CUTE: Measuring LLMs' Understanding of Their Tokenscited by 28
  6. 2022ByGPT5: End-to-End Style-conditioned Poetry Generation with Token-free Language Modelscited by 25
  7. 2022Inducing Character-level Structure in Subword-based Language Models with Type-level Interchange Intervention Trainingcited by 16
  8. 2024Evaluating Morphological Compositional Generalization in Large Language Modelscited by 16
  9. 2023Are Character-level Translations Worth the Wait? Comparing ByT5 and mT5 for Machine Translationcited by 14
  10. 2023Effects of sub-word segmentation on performance of transformer language modelscited by 12
  11. 2025Tokenization is Sensitive to Language Variationcited by 12
  12. 2024Large Vocabulary Size Improves Large Language Modelscited by 11
15 more
  1. 2022Subword-Delimited Downsampling for Better Character-Level Translationcited by 7
  2. 2023Subword Segmental Machine Translation: Unifying Segmentation and Target Sentence Generationcited by 7
  3. 2023Byte-Level Grammatical Error Correction Using Synthetic and Curated Corporacited by 7
  4. 2023Exploring the Impact of Training Data Distribution and Subword Tokenization on Gender Bias in Machine Translationcited by 7
  5. 2025Overcoming Vocabulary Constraints with Pixel-level Fallbackcited by 6
  6. 2022Local Byte Fusion for Neural Machine Translationcited by 5
  7. 2023Assessing the Importance of Frequency versus Compositionality for Subword-based Tokenization in NMTcited by 4
  8. 2025UTF-8 Plumbing: Byte-level Tokenizers Unavoidably Enable LLMs to Generate Ill-formed UTF-8cited by 3
  9. 2026Syllabic Agglutinative Tokenizations for Indonesian LLM: A Study from Gasing Literacy Learning Systemcited by 2
  10. 2023CharSpan: Utilizing Lexical Similarity to Enable Zero-Shot Machine Translation for Extremely Low-resource Languagescited by 1
  11. 2025Improving Language and Modality Transfer in Translation by Character-level Modelingcited by 1
  12. 2023Character-level NMT and language similaritycited by 0
  13. 2024Making Sentence Embeddings Robust to User-Generated Contentcited by 0
  14. 2026The Query Channel: Information-Theoretic Limits of Masking-Based Explanationscited by 0
  15. 2026Toppling the Hierarchy in Byte-level Language Modelingcited by 0