Data to 7 Oct 2026

Enriching Word Vectors with Subword Information

arXiv 1607.04606 ↗First posted 15 Jul 2016cs.CL cs.LGcited by 1,578

Not on the map. Why?

Abstract (arXiv)

Continuous word representations, trained on large unlabeled corpora are useful for many natural language processing tasks. Popular models that learn such representations ignore the morphology of words, by assigning a distinct vector to each word. This is a limitation, especially for languages with large vocabularies and many rare words. In this paper, we propose a new approach based on the skipgram model, where each word is represented as a bag of character n-grams. A vector representation is associated to each character n-gram; words being represented as the sum of these representations. Our method is fast, allowing to train models on large corpora quickly and allows us to compute word representations for words that did not appear in the training data. We evaluate our word representations on nine different languages, both on word similarity and analogy tasks. By comparing to recently proposed morphological word representations, we show that our vectors achieve state-of-the-art performance on these tasks.

Key facts. Cited by 1,578 papers in the map (data through 7 Oct 2026). First posted on arXiv 15 Jul 2016.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2019: 1; 2020: 0; 2021: 451; 2022: 325; 2023: 303; 2024: 223; 2025: 164; 2026: 111.

1,578 papers.

Sort:

By area:

2023Llama 2: Open Foundation and Fine-Tuned Chat Modelscited by 12,865
2021VideoCLIP: Contrastive Pre-training for Zero-shot Video-Text Understandingcited by 596
2023Eliciting Latent Predictions from Transformers with the Tuned Lenscited by 579
2024The Faiss librarycited by 370
2021Self-Diagnosis and Self-Debiasing: A Proposal for Reducing Corpus-Based Bias in NLPcited by 336
2021Membership Inference Attacks on Machine Learning: A Surveycited by 306
2023The Belebele Benchmark: a Parallel Reading Comprehension Dataset in 122 Language Variantscited by 300
2024HelpSteer2: Open-source dataset for training top-performing reward modelscited by 232
2022ADBench: Anomaly Detection Benchmarkcited by 230
2021CANINE: Pre-training an Efficient Tokenization-Free Encoder for Language Representationcited by 229
2021Get Your Vitamin C! Robust Fact Verification with Contrastive Evidencecited by 222
2023A Comprehensive Survey of AI-Generated Content (AIGC): A History of Generative AI from GAN to ChatGPTcited by 221

38 more

2021QuALITY: Question Answering with Long Input Texts, Yes!cited by 204
2021Generating Datasets with Pretrained Language Modelscited by 192
2023MedCPT: Contrastive Pre-trained Transformers with Large-scale PubMed Search Logs for Zero-shot Biomedical Information Retrievalcited by 182
2019Variational Mixture-of-Experts Autoencoders for Multi-Modal Deep Generative Modelscited by 177
2022BEAT: A Large-Scale Semantic and Emotional Multi-Modal Dataset for Conversational Gestures Synthesiscited by 177
2022Relative representations enable zero-shot latent space communicationcited by 175
2024RAID: A Shared Benchmark for Robust Evaluation of Machine-Generated Text Detectorscited by 157
2021MasakhaNER: Named Entity Recognition for African Languagescited by 155
2021VisualMRC: Machine Reading Comprehension on Document Imagescited by 150
2022Towards a Cleaner Document-Oriented Multilingual Crawled Corpuscited by 143
2023PaperQA: Retrieval-Augmented Generative Agent for Scientific Researchcited by 143
2024Aria-UI: Visual Grounding for GUI Instructionscited by 136
2021End-to-End Referring Video Object Segmentation with Multimodal Transformerscited by 135
2021Between words and characters: A Brief History of Open-Vocabulary Modeling and Tokenization in NLPcited by 130
2024Vision-Language Models for Medical Report Generation and Visual Question Answering: A Reviewcited by 121
2021Can Language Models Encode Perceptual Structure Without Grounding? A Case Study in Colorcited by 118
2022Panoptic Scene Graph Generationcited by 114
2021Learning Graph Embeddings for Compositional Zero-shot Learningcited by 112
2021Dive into Deep Learningcited by 111
2023SeaLLMs -- Large Language Models for Southeast Asiacited by 104
2023EMAGE: Towards Unified Holistic Co-Speech Gesture Generation via Expressive Masked Audio Gesture Modelingcited by 101
2021Open World Compositional Zero-Shot Learningcited by 100
2023Vision-Language Models in Remote Sensing: Current Progress and Future Trendscited by 93
2021Survey of Low-Resource Machine Translationcited by 92
2021WECHSEL: Effective initialization of subword embeddings for cross-lingual transfer of monolingual language modelscited by 89
2024A Survey of Large Language Models for Financial Applications: Progress, Prospects and Challengescited by 86
2024A Survey on Multilingual Large Language Models: Corpora, Alignment, and Biascited by 82
2023A Survey of GPT-3 Family Large Language Models Including ChatGPT and GPT-4cited by 81
2026Conditional Memory via Scalable Lookup: A New Axis of Sparsity for Large Language Modelscited by 78
2021FairFil: Contrastive Neural Debiasing Method for Pretrained Text Encoderscited by 75
2022Neighborhood Contrastive Learning for Scientific Document Representations with Citation Embeddingscited by 75
2021Mitigating Political Bias in Language Models Through Reinforced Calibrationcited by 73
2021NodePiece: Compositional and Parameter-Efficient Representations of Large Knowledge Graphscited by 71
2022TweetNLP: Cutting-Edge Natural Language Processing for Social Mediacited by 71
2021Trankit: A Light-Weight Transformer-based Toolkit for Multilingual Natural Language Processingcited by 68
2021Towards generalisable hate speech detection: a review on obstacles and solutionscited by 66
2023A Comprehensive Review of Data-Driven Co-Speech Gesture Generationcited by 66
2024Fine-Tuned 'Small' LLMs (Still) Significantly Outperform Zero-Shot Generative AI Models in Text Classificationcited by 66

50 most cited of 1,578 All 1,578 (JSON)