mlsanityData to 7 Oct 2026

BPEmb: Tokenization-free Pre-trained Subword Embeddings in 275 Languages

arXiv 1710.02187 ↗First posted 5 Oct 2017cs.CLcited by 52

Not on the map. Why?

Abstract (arXiv)

We present BPEmb, a collection of pre-trained subword unit embeddings in 275 languages, based on Byte-Pair Encoding (BPE). In an evaluation using fine-grained entity typing as testbed, BPEmb performs competitively, and for some languages bet- ter than alternative subword approaches, while requiring vastly fewer resources and no tokenization. BPEmb is available at https://github.com/bheinzerling/bpemb

Key facts. Cited by 52 papers in the map (data through 7 Oct 2026). First posted on arXiv 5 Oct 2017.

Builds on

No reference list yet.

Built on by

Cited by year

Citing papers per year: 2021: 17; 2022: 12; 2023: 12; 2024: 6; 2025: 5.

52 papers.

Sort:

  1. 2023Eliciting Latent Predictions from Transformers with the Tuned Lenscited by 579
  2. 2021Towards generalisable hate speech detection: a review on obstacles and solutionscited by 66
  3. 2022TGANet: Text-guided attention for improved polyp segmentationcited by 59
  4. 2021Are Multilingual Models Effective in Code-Switching?cited by 56
  5. 2023Gloss Attention for Gloss-free Sign Language Translationcited by 50
  6. 2022Annotation Error Detection: Analyzing the Past and Present for a More Coherent Futurecited by 45
  7. 2023Visual Question Answering: A Survey on Techniques and Common Trends in Recent Literaturecited by 20
  8. 2023Large Language Models are Fixated by Red Herrings: Exploring Creative Problem Solving and Einstellung Effect using the Only Connect Wall Datasetcited by 17
  9. 2024Lexically Grounded Subword Segmentationcited by 13
  10. 2021To Augment or Not to Augment? A Comparative Study on Text Augmentation Techniques for Low-Resource NLPcited by 12
  11. 2021Capitalization and Punctuation Restoration: a Surveycited by 12
  12. 2022Imputing Out-of-Vocabulary Embeddings with LOVE Makes Language Models Robust with Little Costcited by 9
38 more
  1. 2021Evaluating Neural Word Embeddings for Sanskritcited by 8
  2. 2021Wine is Not v i n. -- On the Compatibility of Tokenizations Across Languagescited by 8
  3. 2022Detecting Unassimilated Borrowings in Spanish: An Annotated Corpus and Approaches to Modelingcited by 6
  4. 2022AsNER -- Annotated Dataset and Baseline for Assamese Named Entity recognitioncited by 6
  5. 2023Uncovering Challenges of Solving the Continuous Gromov-Wasserstein Problemcited by 6
  6. 2023PWESuite: Phonetic Word Embeddings and Tasks They Facilitatecited by 6
  7. 2025M2SFormer: Multi-Spectral and Multi-Scale Attention with Edge-Aware Difficulty Guidance for Image Forgery Localizationcited by 6
  8. 2022An Assessment of the Impact of OCR Noise on Language Modelscited by 5
  9. 2023SanskritShala: A Neural Sanskrit NLP Toolkit with Web-Based Interface for Pedagogical and Annotation Purposescited by 5
  10. 2023Mukhyansh: A Headline Generation Dataset for Indic Languagescited by 5
  11. 2021On the logistical difficulties and findings of Jopara Sentiment Analysiscited by 4
  12. 2022MUST-VQA: MUltilingual Scene-text VQAcited by 4
  13. 2023You Only Transfer What You Share: Intersection-Induced Graph Transfer Learning for Link Predictioncited by 4
  14. 2023DistillCSE: Distilled Contrastive Learning for Sentence Embeddingscited by 4
  15. 2022Few-shot News Recommendation via Cross-lingual Transfercited by 3
  16. 2024Self-StrAE at SemEval-2024 Task 1: Making Self-Structuring AutoEncoders Learn More With Lesscited by 3
  17. 2024Banyan: Improved Representation Learning with Explicit Structurecited by 3
  18. 2024Signformer is all you need: Towards Edge AI for Sign Languagecited by 3
  19. 2021Optimal Size-Performance Tradeoffs: Weighing PoS Tagger Modelscited by 2
  20. 2021Neural Text Classification and Stacked Heterogeneous Embeddings for Named Entity Recognition in SMM4H 2021cited by 2
  21. 2021Multinational Address Parsing: A Zero-Shot Evaluationcited by 2
  22. 2022AsPOS: Assamese Part of Speech Tagger using Deep Learning Approachcited by 2
  23. 2023Unsupervised Approach to Evaluate Sentence-Level Fluency: Do We Really Need Reference?cited by 2
  24. 2024Part-of-Speech Tagger for Bodo Language using Deep Learning approachcited by 2
  25. 2021Spanish Biomedical and Clinical Language Embeddingscited by 1
  26. 2021Enriched Attention for Robust Relation Extractioncited by 1
  27. 2021Opinion Prediction with User Fingerprintingcited by 1
  28. 2023Simulating News Recommendation Ecosystem for Fun and Profitcited by 1
  29. 2021On the Impact of Knowledge-based Linguistic Annotations in the Quality of Scientific Embeddingscited by 0
  30. 2021Named Entity Recognition and Linking Augmented with Large-Scale Structured Datacited by 0
  31. 2021A Neural Network-Based Linguistic Similarity Measure for Entrainment in Conversationscited by 0
  32. 2021LaoPLM: Pre-trained Language Models for Laocited by 0
  33. 2022Like a bilingual baby: The advantage of visually grounding a bilingual language modelcited by 0
  34. 2022Contrastive Credibility Propagation for Reliable Semi-Supervised Learningcited by 0
  35. 2022Applying Multilingual Models to Question Answering (QA)cited by 0
  36. 2024Transformer-Enhanced Iterative Feedback Mechanism for Polyp Segmentationcited by 0
  37. 2025The Cognate Data Bottleneck in Language Phylogeneticscited by 0
  38. 2025PromotionGo at SemEval-2025 Task 11: A Feature-Centric Framework for Cross-Lingual Multi-Emotion Detection in Short Textscited by 0

50 most cited of 52 All 52 (JSON)