mlsanityData to 7 Oct 2026

Advances in Pre-Training Distributed Word Representations

arXiv 1712.09405 ↗First posted 26 Dec 2017cs.CLcited by 207

Not on the map. Why?

Abstract (arXiv)

Many Natural Language Processing applications nowadays rely on pre-trained word representations estimated from large text corpora such as news collections, Wikipedia and Web Crawl. In this paper, we show how to train high-quality word vector representations by using a combination of known tricks that are however rarely used together. The main result of our work is the new set of publicly available pre-trained models that outperform the current state of the art by a large margin on a number of tasks.

Key facts. Cited by 207 papers in the map (data through 7 Oct 2026). First posted on arXiv 26 Dec 2017.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2020: 1; 2021: 70; 2022: 57; 2023: 35; 2024: 21; 2025: 15; 2026: 8.

207 papers.

Sort:

By area:

  1. 2021Image Segmentation Using Text and Image Promptscited by 509
  2. 2021Efficient Deep Learning: A Survey on Making Deep Learning Models Smaller, Faster, and Bettercited by 153
  3. 2022Towards a Cleaner Document-Oriented Multilingual Crawled Corpuscited by 143
  4. 2023Referring Multi-Object Trackingcited by 91
  5. 2021Fast, Effective, and Self-Supervised: Transforming Masked Language Models into Universal Lexical and Sentence Encoderscited by 88
  6. 2022Clotho-AQA: A Crowdsourced Dataset for Audio Question Answeringcited by 84
  7. 2024CARTE: Pretraining and Transfer for Tabular Learningcited by 69
  8. 2021Towards generalisable hate speech detection: a review on obstacles and solutionscited by 66
  9. 2021Are Multilingual Models Effective in Code-Switching?cited by 56
  10. 2021Understanding Negative Samples in Instance Discriminative Self-supervised Representation Learningcited by 47
  11. 2023Natural Language-Assisted Sign Language Recognitioncited by 46
  12. 2024ALOHa: A New Measure for Hallucination in Captioning Modelscited by 44
38 more
  1. 2024Fantastic Semantics and Where to Find Them: Investigating Which Layers of Generative LLMs Reflect Lexical Semanticscited by 43
  2. 2021RpBERT: A Text-image Relation Propagation-based BERT Model for Multimodal NERcited by 37
  3. 2022L3Cube-HingCorpus and HingBERT: A Code Mixed Hindi-English Dataset and BERT Language Modelscited by 33
  4. 2023Human-Centric Multimodal Machine Learning: Recent Advances and Testbed on AI-based Recruitmentcited by 32
  5. 2021DeepHate: Hate Speech Detection via Multi-Faceted Text Representationscited by 29
  6. 2021L3CubeMahaSent: A Marathi Tweet-based Sentiment Analysis Datasetcited by 28
  7. 2021An Efficient DP-SGD Mechanism for Large Scale NLP Modelscited by 27
  8. 2021B-PROP: Bootstrapped Pre-training with Representative Words Prediction for Ad-hoc Retrievalcited by 25
  9. 2022Gender Bias in Word Embeddings: A Comprehensive Analysis of Frequency, Syntax, and Semanticscited by 24
  10. 2021Incorporating External Knowledge to Enhance Tabular Reasoningcited by 23
  11. 2021Understanding and Countering Stereotypes: A Computational Approach to the Stereotype Content Modelcited by 23
  12. 2021NOPE: A Corpus of Naturally-Occurring Presuppositions in Englishcited by 23
  13. 2022Category-Aware Transformer Network for Better Human-Object Interaction Detectioncited by 22
  14. 2022Automated Audio Captioning: An Overview of Recent Progress and New Challengescited by 22
  15. 2022Twitter Topic Classificationcited by 22
  16. 2022Perplexed by Quality: A Perplexity-based Method for Adult and Harmful Content Detection in Multilingual Heterogeneous Web Datacited by 22
  17. 2021A Semi-Personalized System for User Cold Start Recommendation on Music Streaming Appscited by 21
  18. 2022Robust Region Feature Synthesizer for Zero-Shot Object Detectioncited by 20
  19. 2022Canary Extraction in Natural Language Understanding Modelscited by 20
  20. 2023Evaluating Biased Attitude Associations of Language Models in an Intersectional Contextcited by 20
  21. 2021MuMuQA: Multimedia Multi-Hop News Question Answering via Cross-Media Knowledge Extraction and Groundingcited by 19
  22. 2022A Survey on Aspect-Based Sentiment Classificationcited by 18
  23. 2021A Pseudo-Metric between Probability Distributions based on Depth-Trimmed Regionscited by 17
  24. 2021Predict then Interpolate: A Simple Algorithm to Learn Stable Classifierscited by 16
  25. 2022Learning to Split for Automatic Bias Detectioncited by 16
  26. 2022Hierarchical Memory Learning for Fine-Grained Scene Graph Generationcited by 15
  27. 2025Table Foundation Models: on knowledge pre-training for tabular learningcited by 15
  28. 2021Extending Challenge Sets to Uncover Gender Bias in Machine Translation: Impact of Stereotypical Verbs and Adjectivescited by 14
  29. 2021NEREL: A Russian Dataset with Nested Named Entities, Relations and Eventscited by 13
  30. 2022Attentive Deep Neural Networks for Legal Document Retrievalcited by 13
  31. 2023Multi-Scale Attention for Audio Question Answeringcited by 13
  32. 2021ALL Dolphins Are Intelligent and SOME Are Friendly: Probing BERT for Nouns' Semantic Properties and their Prototypicalitycited by 12
  33. 2022Learning to Follow Instructions in Text-Based Gamescited by 12
  34. 2022Deep Emotion Recognition in Textual Conversations: A Surveycited by 12
  35. 2021Rethinking End-to-End Evaluation of Decomposable Tasks: A Case Study on Spoken Language Understandingcited by 11
  36. 2023HCAM -- Hierarchical Cross Attention Model for Multi-modal Emotion Recognitioncited by 11
  37. 2023Tik-to-Tok: Translating Language Models One Token at a Time: An Embedding Initialization Strategy for Efficient Language Adaptationcited by 11
  38. 2022Perceiving the World: Question-guided Reinforcement Learning for Text-based Gamescited by 10

50 most cited of 207 All 207 (JSON)