mlsanityData to 14 Oct 2026

A Survey on Word Meta-Embedding Learning

arXiv 2204.11660 ↗First posted 25 Apr 2022cs.CL cs.AI cs.LGcited by 7

Not on the map. Why?

Abstract (arXiv)

Meta-embedding (ME) learning is an emerging approach that attempts to learn more accurate word embeddings given existing (source) word embeddings as the sole input. Due to their ability to incorporate semantics from multiple source embeddings in a compact manner with superior performance, ME learning has gained popularity among practitioners in NLP. To the best of our knowledge, there exist no prior systematic survey on ME learning and this paper attempts to fill this need. We classify ME learning methods according to multiple factors such as whether they (a) operate on static or contextualised embeddings, (b) trained in an unsupervised manner or (c) fine-tuned for a particular task/domain. Moreover, we discuss the limitations of existing ME learning methods and highlight potential future research directions.

Key facts. Cited by 7 papers in the map (data through 14 Oct 2026). Builds on 33 of its 68 references that are in the map. First posted on arXiv 25 Apr 2022.

Builds on

33 of 68 references.

References · 33

  1. 2017Attention Is All You Needcited by 60,361
  2. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,399
  3. 2019RoBERTa: A Robustly Optimized BERT Pretraining Approachcited by 13,372
  4. 2015Distilling the Knowledge in a Neural Networkcited by 10,324
  5. 2018GLUE: A Multi-Task Benchmark and Analysis Platform for Natural Language Understandingcited by 4,899
  6. 2013Efficient Estimation of Word Representations in Vector Spacecited by 4,375
  7. 2015Domain-Adversarial Training of Neural Networkscited by 3,044
  8. 2019XLNet: Generalized Autoregressive Pretraining for Language Understandingcited by 2,811
  9. 2019ALBERT: A Lite BERT for Self-supervised Learning of Language Representationscited by 2,790
  10. 2018Deep contextualized word representationscited by 2,290
  11. 2016Man is to Computer Programmer as Woman is to Homemaker? Debiasing Word Embeddingscited by 1,312
  12. 2016Bag of Tricks for Efficient Text Classificationcited by 876
21 more
  1. 2017Word Translation Without Parallel Datacited by 487
  2. 2017All-but-the-Top: Simple and Effective Postprocessing for Word Representationscited by 212
  3. 2018Large scale distributed neural network training through online distillationcited by 146
  4. 2019Gender-preserving Debiasing for Pre-trained Word Embeddingscited by 61
  5. 2018On the Dimensionality of Word Embeddingcited by 46
  6. 2015Efficient Non-parametric Estimation of Multiple Embeddings per Word in Vector Spacecited by 35
  7. 2018Frustratingly Easy Meta-Embedding -- Computing Meta-Embeddings by Averaging Source Word Embeddingscited by 15
  8. 2018Dynamic Meta-Embeddings for Improved Sentence Representationscited by 15
  9. 2018Improving Cross-Lingual Word Embeddings by Meeting in the Middlecited by 13
  10. 2017Think Globally, Embed Locally --- Locally Linear Meta-embedding of Wordscited by 8
  11. 2019Sentence Meta-Embeddings for Unsupervised Semantic Textual Similaritycited by 8
  12. 2019Hierarchical Meta-Embeddings for Code-Switching Named Entity Recognitioncited by 7
  13. 2017Linear Ensembles of Word Embedding Modelscited by 6
  14. 2020Learning Geometric Word Meta-Embeddingscited by 6
  15. 2017Using k-way Co-occurrences for Learning Word Embeddingscited by 5
  16. 2015Joint Word Representation Learning using a Corpus and a Semantic Lexiconcited by 4
  17. 2020FAME: Feature-Based Adversarial Meta-Embeddings for Robust Input Representationscited by 4
  18. 2020Learning Efficient Task-Specific Meta-Embeddings with Word Prismscited by 4
  19. 2022Unsupervised Attention-based Sentence-Level Meta-Embeddings from Contextualised Language Modelscited by 4
  20. 2018Meta-Embedding as Auxiliary Task Regularizationcited by 2
  21. 2020A Common Semantic Space for Monolingual and Cross-Lingual Meta-Embeddingscited by 2

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2022: 2; 2023: 3; 2024: 1; 2025: 0; 2026: 1.

7 papers.

  1. 2022Gender Bias in Meta-Embeddingscited by 6
  2. 2023TADA: Efficient Task-Agnostic Domain Adaptation for Transformerscited by 6
  3. 2022Learning Meta Word Embeddings by Unsupervised Weighted Concatenation of Source Embeddingscited by 5
  4. 2023A Neighbourhood-Aware Differential Privacy Mechanism for Static Word Embeddingscited by 4
  5. 2023Together We Make Sense -- Learning Meta-Sense Embeddings from Pretrained Static Sense Embeddingscited by 1
  6. 2024Stitching Gaps: Fusing Situated Perceptual Knowledge with Vision Transformers for High-Level Image Classificationcited by 1
  7. 2026Tracing the Evolution of Word Embedding Techniques in Natural Language Processingcited by 0