mlsanityData to 7 Oct 2026

MaxMatch-Dropout: Subword Regularization for WordPiece

arXiv 2209.04126 ↗First posted 9 Sep 2022cs.CLcited by 8

Not on the map. Why?

Abstract (arXiv)

We present a subword regularization method for WordPiece, which uses a maximum matching algorithm for tokenization. The proposed method, MaxMatch-Dropout, randomly drops words in a search using the maximum matching algorithm. It realizes finetuning with subword regularization for popular pretrained language models such as BERT-base. The experimental results demonstrate that MaxMatch-Dropout improves the performance of text classification and machine translation tasks as well as other subword regularization methods. Moreover, we provide a comparative analysis of subword regularization methods: subword regularization with SentencePiece (Unigram), BPE-Dropout, and MaxMatch-Dropout.

Key facts. Cited by 8 papers in the map (data through 7 Oct 2026). Builds on 17 of its 29 references that are in the map. First posted on arXiv 9 Sep 2022.

Builds on

17 of 29 references.

References · 17

  1. 2017Attention Is All You Needcited by 59,932
  2. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  3. 2018GLUE: A Multi-Task Benchmark and Analysis Platform for Natural Language Understandingcited by 4,854
  4. 2016SQuAD: 100,000+ Questions for Machine Comprehension of Textcited by 4,115
  5. 2015Neural Machine Translation of Rare Words with Subword Unitscited by 2,898
  6. 2018SentencePiece: A simple and language independent subword tokenizer and detokenizer for Neural Text Processingcited by 1,973
  7. 2018A Call for Clarity in Reporting BLEU Scorescited by 1,790
  8. 2019fairseq: A Fast, Extensible Toolkit for Sequence Modelingcited by 1,486
  9. 2018Subword Regularization: Improving Neural Network Translation Models with Multiple Subword Candidatescited by 632
  10. 2016Ups and Downs: Modeling the Visual Evolution of Fashion Trends with One-Class Collaborative Filteringcited by 517
  11. 2019BPE-Dropout: Simple and Effective Subword Regularizationcited by 189
  12. 2020Byte Pair Encoding is Suboptimal for Language Model Pretrainingcited by 189
5 more
  1. 2021KLUE: Korean Language Understanding Evaluationcited by 113
  2. 2020Fast WordPiece Tokenizationcited by 85
  3. 2020Evaluating Robustness to Input Perturbations for Neural Machine Translationcited by 36
  4. 2021Joint Optimization of Tokenization and Downstream Modelcited by 15
  5. 2022Single Model Ensemble for Subword Regularized Models in Low-Resource Machine Translationcited by 2

Built on by

Cited by year

Citing papers per year: 2022: 1; 2023: 2; 2024: 3; 2025: 2.

8 papers.

  1. 2022Incorporating Context into Subword Vocabulariescited by 11
  2. 2024We're Calling an Intervention: Exploring Fundamental Hurdles in Adapting Language Models to Nonstandard Textcited by 6
  3. 2025StochasTok: Improving Fine-Grained Subword Understanding in LLMscited by 6
  4. 2025Entropy-Guided Token Dropout: Training Autoregressive Language Models with Limited Domain Datacited by 5
  5. 2024Distributional Properties of Subword Regularizationcited by 4
  6. 2024SubRegWeigh: Effective and Efficient Annotation Weighing with Subword Regularizationcited by 1
  7. 2023Downstream Task-Oriented Neural Tokenizer Optimization with Vocabulary Restriction as Post Processingcited by 0
  8. 2023Tokenization Preference for Human and Machine Learning Model: An Annotation Studycited by 0