mlsanityData to 7 Oct 2026

What changes when you randomly choose BPE merge operations? Not much

arXiv 2305.03029 ↗First posted 4 May 2023cs.CL cs.AIcited by 10

Not on the map. Why?

Abstract (arXiv)

We introduce three simple randomized variants of byte pair encoding (BPE) and explore whether randomizing the selection of merge operations substantially affects a downstream machine translation task. We focus on translation into morphologically rich languages, hypothesizing that this task may show sensitivity to the method of choosing subwords. Analysis using a Bayesian linear model indicates that two of the variants perform nearly indistinguishably compared to standard BPE while the other degrades performance less than we anticipated. We conclude that although standard BPE is widely used, there exists an interesting universe of potential variations on it worth investigating. Our code is available at: https://github.com/bltlab/random-bpe.

Key facts. Cited by 10 papers in the map (data through 7 Oct 2026). Builds on 13 of its 23 references that are in the map. First posted on arXiv 4 May 2023.

Builds on

13 of 23 references.

References · 13

  1. 2014Learning Phrase Representations using RNN Encoder-Decoder for Statistical Machine Translationcited by 4,016
  2. 2014Sequence to Sequence Learning with Neural Networkscited by 2,974
  3. 2015Neural Machine Translation of Rare Words with Subword Unitscited by 2,898
  4. 2018SentencePiece: A simple and language independent subword tokenizer and detokenizer for Neural Text Processingcited by 1,973
  5. 2018A Call for Clarity in Reporting BLEU Scorescited by 1,790
  6. 2016Google's Neural Machine Translation System: Bridging the Gap between Human and Machine Translationcited by 1,471
  7. 2015Effective Approaches to Attention-based Neural Machine Translationcited by 1,024
  8. 2018Subword Regularization: Improving Neural Network Translation Models with Multiple Subword Candidatescited by 632
  9. 2021Between words and characters: A Brief History of Open-Vocabulary Modeling and Tokenization in NLPcited by 130
  10. 2020Fast WordPiece Tokenizationcited by 85
  11. 2019Revisiting Low-Resource Neural Machine Translation: A Case Studycited by 61
  12. 2020Dynamic Programming Encoding for Subword Segmentation in Neural Machine Translationcited by 38
1 more
  1. 2021Evaluating Multiway Multilingual NMT in the Turkic Languagescited by 8

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 1; 2024: 4; 2025: 3; 2026: 2.

10 papers.

  1. 2024Tokenization Is More Than Compressioncited by 87
  2. 2024Unpacking Tokenization: Evaluating Text Compression and its Correlation with Model Performancecited by 61
  3. 2023Tokenization and the Noiseless Channelcited by 58
  4. 2024Greed is All You Need: An Evaluation of Tokenizer Inference Methodscited by 34
  5. 2025Beyond Text Compression: Evaluating Tokenizers Across Scalescited by 11
  6. 2024When Every Token Counts: Optimal Segmentation for Low-Resource Language Modelscited by 8
  7. 2026LiteToken: Removing Intermediate Merge Residues From BPE Tokenizerscited by 3
  8. 2025DPAR: Dynamic Patchification for Efficient Autoregressive Visual Generationcited by 2
  9. 2025Train It and Forget It: Merge Lists are Unnecessary for BPE Inference in Language Modelscited by 1
  10. 2026How Long Is a Piece of String? A Brief Empirical Analysis of Tokenizerscited by 0