mlsanityData to 7 Oct 2026

Frustratingly Easy Meta-Embedding -- Computing Meta-Embeddings by Averaging Source Word Embeddings

arXiv 1804.05262 ↗First posted 14 Apr 2018cs.CLcited by 15

Not on the map. Why?

Abstract (arXiv)

Creating accurate meta-embeddings from pre-trained source embeddings has received attention lately. Methods based on global and locally-linear transformation and concatenation have shown to produce accurate meta-embeddings. In this paper, we show that the arithmetic mean of two distinct word embedding sets yields a performant meta-embedding that is comparable or better than more complex meta-embedding learning methods. The result seems counter-intuitive given that vector spaces in different source embeddings are not comparable and cannot be simply averaged. We give insight into why averaging can still produce accurate meta-embedding despite the incomparability of the source vector spaces.

Key facts. Cited by 15 papers in the map (data through 7 Oct 2026). First posted on arXiv 14 Apr 2018.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 5; 2022: 5; 2023: 2; 2024: 1; 2025: 0; 2026: 2.

15 papers.

Sort:

  1. 2021Are Multilingual Models Effective in Code-Switching?cited by 56
  2. 2022A Survey on Word Meta-Embedding Learningcited by 7
  3. 2022Gender Bias in Meta-Embeddingscited by 6
  4. 2023TADA: Efficient Task-Agnostic Domain Adaptation for Transformerscited by 6
  5. 2022Learning Meta Word Embeddings by Unsupervised Weighted Concatenation of Source Embeddingscited by 5
  6. 2021Are Word Embedding Methods Stable and Should We Care About It?cited by 4
  7. 2022Unsupervised Attention-based Sentence-Level Meta-Embeddings from Contextualised Language Modelscited by 4
  8. 2021A Meta-embedding-based Ensemble Approach for ICD Coding Predictioncited by 2
  9. 2021Denoising Word Embeddings by Averaging in a Shared Spacecited by 2
  10. 2023Together We Make Sense -- Learning Meta-Sense Embeddings from Pretrained Static Sense Embeddingscited by 1
  11. 2024From Latent to Engine Manifolds: Analyzing ImageBind's Multimodal Embedding Spacecited by 1
  12. 2021KARL-Trans-NER: Knowledge Aware Representation Learning for Named Entity Recognition using Transformerscited by 0
3 more
  1. 2022Taxonomy Enrichment with Text and Graph Vector Representationscited by 0
  2. 2026Integrating Meta-Features with Knowledge Graph Embeddings for Meta-Learningcited by 0
  3. 2026Where Do Multilingual Vision-Language Encoders Fail on Low-Resource Languages?cited by 0