# Concatenated Power Mean Word Embeddings as Universal Cross-Lingual Sentence Representations

arXiv: https://arxiv.org/abs/1803.01400 · HTML: https://mlsanity.com/paper/1803.01400/

## Key facts

- Cited by 20 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 4 Mar 2018.

## Built on by (most cited first, 20 of 20)

- [Gloss Attention for Gloss-free Sign Language Translation](https://mlsanity.com/paper/2307.07361/) (2023)
- [Automatic Text Evaluation through the Lens of Wasserstein Barycenters](https://mlsanity.com/paper/2108.12463/) (2021)
- [InfoLM: A New Metric to Evaluate Summarization & Data2Text Generation](https://mlsanity.com/paper/2112.01589/) (2021)
- [Survey on reinforcement learning for language processing](https://mlsanity.com/paper/2104.05565/) (2021)
- [Just Rank: Rethinking Evaluation with Word and Sentence Similarities](https://mlsanity.com/paper/2203.02679/) (2022)
- [Unsupervised Layer-wise Score Aggregation for Textual OOD Detection](https://mlsanity.com/paper/2302.09852/) (2023)
- [Innovations in Neural Data-to-text Generation: A Survey](https://mlsanity.com/paper/2207.12571/) (2022)
- [Reproducibility Issues for BERT-based Evaluation Metrics](https://mlsanity.com/paper/2204.00004/) (2022)
- [QRelScore: Better Evaluating Generated Questions with Deeper Understanding of Context-aware Relevance](https://mlsanity.com/paper/2204.13921/) (2022)
- [Learning Zero-Shot Multifaceted Visually Grounded Word Embeddings via Multi-Task Training](https://mlsanity.com/paper/2104.07500/) (2021)
- [Codenames as a Benchmark for Large Language Models](https://mlsanity.com/paper/2412.11373/) (2024)
- [Unsupervised Sentence-embeddings by Manifold Approximation and Projection](https://mlsanity.com/paper/2102.03795/) (2021)
- [SWE2: SubWord Enriched and Significant Word Emphasized Framework for Hate Speech Detection](https://mlsanity.com/paper/2409.16673/) (2024)
- [Beyond Mahalanobis-Based Scores for Textual OOD Detection](https://mlsanity.com/paper/2211.13527/) (2022)
- [Discrete Cosine Transform as Universal Sentence Encoder](https://mlsanity.com/paper/2106.00934/) (2021)
- [A Novel Perspective for Positive-Unlabeled Learning via Noisy Labels](https://mlsanity.com/paper/2103.04685/) (2021)
- [LiteLMGuard: Seamless and Lightweight On-Device Prompt Filtering for Safeguarding Small Language Models against Quantization-induced Risks and Vulnerabilities](https://mlsanity.com/paper/2505.05619/) (2025)
- [NoPPA: Non-Parametric Pairwise Attention Random Walk Model for Sentence Representation](https://mlsanity.com/paper/2302.12903/) (2023)
- [Sentiment analysis in Tourism: Fine-tuning BERT or sentence embeddings concatenation?](https://mlsanity.com/paper/2312.07797/) (2023)
- [Investigating Consistency in Query-Based Meeting Summarization: A Comparative Study of Different Embedding Methods](https://mlsanity.com/paper/2402.06907/) (2024)

## Abstract (arXiv)

Average word embeddings are a common baseline for more sophisticated sentence embedding techniques. However, they typically fall short of the performances of more complex models such as InferSent. Here, we generalize the concept of average word embeddings to power mean word embeddings. We show that the concatenation of different types of power mean word embeddings considerably closes the gap to state-of-the-art methods monolingually and substantially outperforms these more complex techniques cross-lingually. In addition, our proposed method outperforms different recently proposed baselines such as SIF and Sent2Vec by a solid margin, thus constituting a much harder-to-beat monolingual baseline. Our data and code are publicly available.