mlsanityData to 7 Oct 2026

Neural Machine Translation of Rare Words with Subword Units

arXiv 1508.07909 ↗First posted 31 Aug 2015cs.CLcited by 2,898

Not on the map. Why?

Cited by 55 in Multimodal LLMs / vision-language models, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

Neural machine translation (NMT) models typically operate with a fixed vocabulary, but translation is an open-vocabulary problem. Previous work addresses the translation of out-of-vocabulary words by backing off to a dictionary. In this paper, we introduce a simpler and more effective approach, making the NMT model capable of open-vocabulary translation by encoding rare and unknown words as sequences of subword units. This is based on the intuition that various word classes are translatable via smaller units than words, for instance names (via character copying or transliteration), compounds (via compositional translation), and cognates and loanwords (via phonological and morphological transformations). We discuss the suitability of different word segmentation techniques, including simple character n-gram models and a segmentation based on the byte pair encoding compression algorithm, and empirically show that subword models improve over a back-off dictionary baseline for the WMT 15 translation tasks English-German and English-Russian by 1.1 and 1.3 BLEU, respectively.

Key facts. Cited by 2,898 papers in the map (data through 7 Oct 2026). First posted on arXiv 31 Aug 2015.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2018: 2; 2019: 1; 2020: 2; 2021: 612; 2022: 511; 2023: 520; 2024: 499; 2025: 407; 2026: 344.

2,898 papers.

Sort:

By area:

  1. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
  2. 2023LLaMA: Open and Efficient Foundation Language Modelscited by 13,958
  3. 2023Llama 2: Open Foundation and Fine-Tuned Chat Modelscited by 12,865
  4. 2025Qwen3 Technical Reportcited by 8,195
  5. 2021RoFormer: Enhanced Transformer with Rotary Position Embeddingcited by 4,581
  6. 2021Zero-Shot Text-to-Image Generationcited by 4,276
  7. 2022Robust Speech Recognition via Large-Scale Weak Supervisioncited by 3,972
  8. 2022OPT: Open Pre-trained Transformer Language Modelscited by 3,464
  9. 2023Grounding DINO: Marrying DINO with Grounded Pre-Training for Open-Set Object Detectioncited by 3,274
  10. 2023A Survey of Large Language Modelscited by 2,537
  11. 2021CLIPScore: A Reference-free Evaluation Metric for Image Captioningcited by 2,388
  12. 2021BEiT: BERT Pre-Training of Image Transformerscited by 2,360
38 more
  1. 2021Learning to Prompt for Vision-Language Modelscited by 2,269
  2. 2019Fine-Tuning Language Models from Human Preferencescited by 2,124
  3. 2023Code Llama: Open Foundation Models for Codecited by 2,105
  4. 2022Scaling Autoregressive Models for Content-Rich Text-to-Image Generationcited by 1,287
  5. 2022CoCa: Contrastive Captioners are Image-Text Foundation Modelscited by 1,265
  6. 2021DeBERTaV3: Improving DeBERTa using ELECTRA-Style Pre-Training with Gradient-Disentangled Embedding Sharingcited by 1,219
  7. 2022LaMDA: Language Models for Dialog Applicationscited by 1,218
  8. 2024DeepSeek-Coder: When the Large Language Model Meets Programming -- The Rise of Code Intelligencecited by 1,154
  9. 2024ChatGLM: A Family of Large Language Models from GLM-130B to GLM-4 All Toolscited by 1,096
  10. 2021A General Language Assistant as a Laboratory for Alignmentcited by 1,023
  11. 2023BloombergGPT: A Large Language Model for Financecited by 923
  12. 2024Chameleon: Mixed-Modal Early-Fusion Foundation Modelscited by 904
  13. 2021iBOT: Image BERT Pre-Training with Online Tokenizercited by 891
  14. 2021CodeT5: Identifier-aware Unified Pre-trained Encoder-Decoder Models for Code Understanding and Generationcited by 890
  15. 2024DeepSeekMoE: Towards Ultimate Expert Specialization in Mixture-of-Experts Language Modelscited by 834
  16. 2021MDETR -- Modulated Detection for End-to-End Multi-Modal Understandingcited by 811
  17. 2021Vector Quantized Diffusion Model for Text-to-Image Synthesiscited by 773
  18. 2022OFA: Unifying Architectures, Tasks, and Modalities Through a Simple Sequence-to-Sequence Learning Frameworkcited by 742
  19. 2021CodeXGLUE: A Machine Learning Benchmark Dataset for Code Understanding and Generationcited by 732
  20. 2022data2vec: A General Framework for Self-supervised Learning in Speech, Vision and Languagecited by 718
  21. 2024Chronos: Learning the Language of Time Seriescited by 712
  22. 2022Autoregressive Image Generation using Residual Quantizationcited by 706
  23. 2024MiniCPM: Unveiling the Potential of Small Language Models with Scalable Training Strategiescited by 694
  24. 2021FLAVA: A Foundational Language And Vision Alignment Modelcited by 646
  25. 2025FAST: Efficient Action Tokenization for Vision-Language-Action Modelscited by 643
  26. 2023A Comprehensive Overview of Large Language Modelscited by 639
  27. 2021FILIP: Fine-grained Interactive Language-Image Pre-Trainingcited by 626
  28. 2021Perceiver IO: A General Architecture for Structured Inputs & Outputscited by 616
  29. 2021XLS-R: Self-supervised Cross-lingual Speech Representation Learning at Scalecited by 589
  30. 2022BioGPT: Generative Pre-trained Transformer for Biomedical Text Generation and Miningcited by 531
  31. 2022Winoground: Probing Vision and Language Models for Visio-Linguistic Compositionalitycited by 523
  32. 2023BiomedCLIP: a multimodal biomedical foundation model pretrained from fifteen million scientific image-text pairscited by 506
  33. 2021ByT5: Towards a token-free future with pre-trained byte-to-byte modelscited by 504
  34. 2022InCoder: A Generative Model for Code Infilling and Synthesiscited by 492
  35. 2022Make-A-Scene: Scene-Based Text-to-Image Generation with Human Priorscited by 487
  36. 2022LayoutLMv3: Pre-training for Document AI with Unified Text and Image Maskingcited by 481
  37. 2021Linear Transformers Are Secretly Fast Weight Programmerscited by 471
  38. 2022DiffuSeq: Sequence to Sequence Text Generation with Diffusion Modelscited by 439

50 most cited of 2,898 All 2,898 (JSON)