# Frustratingly Easy Meta-Embedding -- Computing Meta-Embeddings by Averaging Source Word Embeddings

arXiv: https://arxiv.org/abs/1804.05262 · HTML: https://mlsanity.com/paper/1804.05262/

## Key facts

- Cited by 15 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 14 Apr 2018.

## Built on by (most cited first, 15 of 15)

- [Are Multilingual Models Effective in Code-Switching?](https://mlsanity.com/paper/2103.13309/) (2021)
- [A Survey on Word Meta-Embedding Learning](https://mlsanity.com/paper/2204.11660/) (2022)
- [Gender Bias in Meta-Embeddings](https://mlsanity.com/paper/2205.09867/) (2022)
- [TADA: Efficient Task-Agnostic Domain Adaptation for Transformers](https://mlsanity.com/paper/2305.12717/) (2023)
- [Learning Meta Word Embeddings by Unsupervised Weighted Concatenation of Source Embeddings](https://mlsanity.com/paper/2204.12386/) (2022)
- [Are Word Embedding Methods Stable and Should We Care About It?](https://mlsanity.com/paper/2104.08433/) (2021)
- [Unsupervised Attention-based Sentence-Level Meta-Embeddings from Contextualised Language Models](https://mlsanity.com/paper/2204.07746/) (2022)
- [A Meta-embedding-based Ensemble Approach for ICD Coding Prediction](https://mlsanity.com/paper/2102.13622/) (2021)
- [Denoising Word Embeddings by Averaging in a Shared Space](https://mlsanity.com/paper/2106.02954/) (2021)
- [Together We Make Sense -- Learning Meta-Sense Embeddings from Pretrained Static Sense Embeddings](https://mlsanity.com/paper/2305.19092/) (2023)
- [From Latent to Engine Manifolds: Analyzing ImageBind's Multimodal Embedding Space](https://mlsanity.com/paper/2409.10528/) (2024)
- [KARL-Trans-NER: Knowledge Aware Representation Learning for Named Entity Recognition using Transformers](https://mlsanity.com/paper/2111.15436/) (2021)
- [Taxonomy Enrichment with Text and Graph Vector Representations](https://mlsanity.com/paper/2201.08598/) (2022)
- [Integrating Meta-Features with Knowledge Graph Embeddings for Meta-Learning](https://mlsanity.com/paper/2603.19888/) (2026)
- [Where Do Multilingual Vision-Language Encoders Fail on Low-Resource Languages?](https://mlsanity.com/paper/2608.30725/) (2026)

## Abstract (arXiv)

Creating accurate meta-embeddings from pre-trained source embeddings has received attention lately. Methods based on global and locally-linear transformation and concatenation have shown to produce accurate meta-embeddings. In this paper, we show that the arithmetic mean of two distinct word embedding sets yields a performant meta-embedding that is comparable or better than more complex meta-embedding learning methods. The result seems counter-intuitive given that vector spaces in different source embeddings are not comparable and cannot be simply averaged. We give insight into why averaging can still produce accurate meta-embedding despite the incomparability of the source vector spaces.