# Enhancing Sentence Embedding with Generalized Pooling

arXiv: https://arxiv.org/abs/1806.09828 · HTML: https://mlsanity.com/paper/1806.09828/

## Key facts

- Cited by 8 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 26 Jun 2018.

## Built on by (most cited first, 8 of 8)

- [A Picture is Worth More Than 77 Text Tokens: Evaluating CLIP-Style Models on Dense Captions](https://mlsanity.com/paper/2312.08578/) (2023)
- [On Training Instance Selection for Few-Shot Neural Text Generation](https://mlsanity.com/paper/2107.03176/) (2021)
- [SetCSE: Set Operations using Contrastive Learning of Sentence Embeddings](https://mlsanity.com/paper/2404.17606/) (2024)
- [Russian News Clustering and Headline Selection Shared Task](https://mlsanity.com/paper/2105.00981/) (2021)
- [Humanly Certifying Superhuman Classifiers](https://mlsanity.com/paper/2109.07867/) (2021)
- [Explain and Conquer: Personalised Text-based Reviews to Achieve Transparency](https://mlsanity.com/paper/2205.01759/) (2022)
- [SENS-ASR: Semantic Embedding injection in Neural-transducer for Streaming Automatic Speech Recognition](https://mlsanity.com/paper/2603.10005/) (2026)
- [GATech at AbjadMed: Bidirectional Encoders vs. Causal Decoders: Insights from 82-Class Arabic Medical Classification](https://mlsanity.com/paper/2603.10008/) (2026)

## Abstract (arXiv)

Pooling is an essential component of a wide variety of sentence representation and embedding models. This paper explores generalized pooling methods to enhance sentence embedding. We propose vector-based multi-head attention that includes the widely used max pooling, mean pooling, and scalar self-attention as special cases. The model benefits from properly designed penalization terms to reduce redundancy in multi-head attention. We evaluate the proposed model on three different tasks: natural language inference (NLI), author profiling, and sentiment classification. The experiments show that the proposed model achieves significant improvement over strong sentence-encoding-based methods, resulting in state-of-the-art performances on four datasets. The proposed approach can be easily implemented for more problems than we discuss in this paper.