SentEval: An Evaluation Toolkit for Universal Sentence Representations
Not on the map. Why?
Abstract (arXiv)
We introduce SentEval, a toolkit for evaluating the quality of universal sentence representations. SentEval encompasses a variety of tasks, including binary and multi-class classification, natural language inference and sentence similarity. The set of tasks was selected based on what appears to be the community consensus regarding the appropriate evaluations for universal sentence representations. The toolkit comes with scripts to download and preprocess datasets, and an easy interface to evaluate sentence encoders. The aim is to provide a fairer, less cumbersome and more centralized way for evaluating sentence representations.
Key facts. Cited by 294 papers in the map (data through 7 Oct 2026). First posted on arXiv 14 Mar 2018.
Builds on
No reference list yet.
Built on by
Citing papers per year: 2021: 60; 2022: 70; 2023: 75; 2024: 55; 2025: 20; 2026: 14.
294 papers.
Sort:
By area:
- 2021SimCSE: Simple Contrastive Learning of Sentence Embeddingscited by 2,168
- 2022Text Embeddings by Weakly-Supervised Contrastive Pre-trainingcited by 1,216
- 2023AGIEval: A Human-Centric Benchmark for Evaluating Foundation Modelscited by 701
- 2022MTEB: Massive Text Embedding Benchmarkcited by 623
- 2023C-Pack: Packed Resources For General Chinese Embeddingscited by 556
- 2021Sentence-T5: Scalable Sentence Encoders from Pre-trained Text-to-Text Modelscited by 462
- 2021Dynabench: Rethinking Benchmarking in NLPcited by 447
- 2023Larger language models do in-context learning differentlycited by 386
- 2022Text and Code Embeddings by Contrastive Pre-Trainingcited by 358
- 2021AI and the Everything in the Whole Wide World Benchmarkcited by 340
- 2022One Embedder, Any Task: Instruction-Finetuned Text Embeddingscited by 329
- 2021LexGLUE: A Benchmark Dataset for Legal Language Understanding in Englishcited by 302
38 more
- 2021Parameter-efficient Multi-task Fine-tuning for Transformers via Shared Hypernetworkscited by 278
- 2021ConSERT: A Contrastive Framework for Self-Supervised Sentence Representation Transfercited by 231
- 2024Generative Representational Instruction Tuningcited by 227
- 2021Masked Language Modeling and the Distributional Hypothesis: Order Word Matters Pre-training for Littlecited by 176
- 2021Condenser: a Pre-training Architecture for Dense Retrievalcited by 162
- 2022Efficient Few-Shot Learning Without Promptscited by 135
- 2022DiffCSE: Difference-based Contrastive Learning for Sentence Embeddingscited by 131
- 2021BERT: A Review of Applications in Natural Language Processing and Understandingcited by 109
- 2021Self-Guided Contrastive Learning for BERT Sentence Representationscited by 103
- 2021CBLUE: A Chinese Biomedical Language Understanding Evaluation Benchmarkcited by 99
- 2023AnglE-optimized Text Embeddingscited by 98
- 2021VALUE: A Multi-Task Benchmark for Video-and-Language Understanding Evaluationcited by 95
- 2022Same Author or Just Same Topic? Towards Content-Independent Style Representationscited by 83
- 2023Scaling Sentence Embeddings with Large Language Modelscited by 83
- 2022Continual Pre-Training Mitigates Forgetting in Language and Visioncited by 82
- 2023Sentence Embedding Leaks More Information than You Expect: Generative Embedding Inversion Attack to Recover the Whole Sentencecited by 81
- 2022PromptBERT: Improving BERT Sentence Embeddings with Promptscited by 75
- 2023Symbol tuning improves in-context learning in language modelscited by 72
- 2023What Do Self-Supervised Speech Models Know About Words?cited by 62
- 2021BERT Learns to Teach: Knowledge Distillation with Meta Learningcited by 61
- 2022Debiased Contrastive Learning of Unsupervised Sentence Representationscited by 56
- 2022Parameter-Efficient and Student-Friendly Knowledge Distillationcited by 47
- 2021Towards Efficient NLP: A Standard Evaluation and A Strong Baselinecited by 44
- 2024Why Larger Language Models Do In-context Learning Differently?cited by 44
- 2021On the proper role of linguistically-oriented deep net analysis in linguistic theorizingcited by 43
- 2021Adversarial Training Helps Transfer Learning via Better Representationscited by 36
- 2023How Predictable Are Large Language Model Capabilities? A Case Study on BIG-benchcited by 34
- 2021STraTA: Self-Training with Task Augmentation for Better Few-shot Learningcited by 33
- 2021The Inductive Bias of In-Context Learning: Rethinking Pretraining Example Designcited by 33
- 2022A Data-Based Perspective on Transfer Learningcited by 32
- 2023BeLLM: Backward Dependency Enhanced Large Language Model for Sentence Embeddingscited by 32
- 2021Pairwise Supervised Contrastive Learning of Sentence Representationscited by 31
- 2022MABEL: Attenuating Gender Bias using Textual Entailment Datacited by 31
- 2022Improved Universal Sentence Embeddings with Prompt-based Contrastive Learning and Energy-based Learningcited by 30
- 2023Instruction-following Evaluation through Verbalizer Manipulationcited by 30
- 2021Exploring the Role of BERT Token Representations to Explain Sentence Probing Resultscited by 29
- 2022PCL: Peer-Contrastive Learning with Diverse Augmentations for Unsupervised Sentence Embeddingscited by 29
- 2023Improving Contrastive Learning of Sentence Embeddings from AI Feedbackcited by 29
50 most cited of 294 All 294 (JSON)
- 2026AISA: AI Safety Assistant Framework for Continuous Improvement of Highway Construction17 Aug 2026
- 2026MTEB-BR: A Text Embedding Benchmark for Brazilian Portuguese6 Jul 2026
- 2026When AUC 0.998 Is Not Enough: A Candidate Evaluation Protocol for Hidden-State Probes of Indirect Prompt Injection in Multimodal Computer-Use Agents22 Jun 2026
- 2026TALAS: Teacher-Anchored Layer Alignment with Adaptive Sharpness-Aware Minimization for Embedding Distillation20 Jun 2026
- 2026Cross-Modal Masked Compositional Concept Modeling for Enhancing Visio-Linguistic Compositionality11 Jun 2026
- 2026MIC: Maximizing Informational Capacity in Adaptive Representations via Isotropic Subspace Alignment28 May 2026
- 2026Hubness, Not Anisotropy, Drives Cross-Lingual Retrieval Asymmetry in Multilingual Embedding Models26 May 2026
- 2026Universal Activation Verbalizer: A Unified Framework for Cross-Model Activation Explanation25 May 2026
- 2026MIPIC: Matryoshka Representation Learning via Self-Distilled Intra-Relational and Progressive Information Chaining27 Apr 2026
- 2026Predict and Reconstruct: Joint Objectives for Self-Supervised Language Representation Learning16 Apr 2026
- 2026Computational Lesions in Multilingual Language Models Separate Shared and Language-specific Brain Alignment12 Apr 2026
- 2026Understanding LLM Performance Degradation in Multi-Instance Processing: The Roles of Instance Count and Context Length23 Mar 2026
38 more
- 2026LMEB: Long-horizon Memory Embedding Benchmark13 Mar 2026
- 2026Bipartite Mode Matching for Vision Training Set Search from a Hierarchical Data Server14 Jan 2026
- 2025Instruction-Following Evaluation of Large Vision-Language Models29 Dec 2025
- 2025Advances and Challenges in Semantic Textual Similarity: A Comprehensive Survey19 Dec 2025
- 2025AncientBench: Towards Comprehensive Evaluation on Excavated and Transmitted Chinese Corpora19 Dec 2025
- 2025Convolutional Lie Operator for Sentence Classification18 Dec 2025
- 2025AttnCache: Accelerating Self-Attention Inference for LLM Prefill via Attention Cache29 Oct 2025
- 2025SEA-BED: How Do Embedding Models Represent Southeast Asian Languages?17 Aug 2025
- 2025Combining Discrete Wavelet and Cosine Transforms for Efficient Sentence Embedding1 Aug 2025
- 2025Improving Multimodal Contrastive Learning of Sentence Embeddings with Object-Phrase Alignment1 Aug 2025
- 2025Semantic Compression for Word and Sentence Embeddings using Discrete Wavelet Transform31 Jul 2025
- 2025VN-MTEB: Vietnamese Massive Text Embedding Benchmark29 Jul 2025
- 2025On The Role of Pretrained Language Models in General-Purpose Text Embeddings: A Survey28 Jul 2025
- 2025DALR: Dual-level Alignment Learning for Multimodal Sentence Representation Learning26 Jun 2025
- 2025Illusion or Algorithm? Investigating Memorization, Emergence, and Symbolic Processing in In-Context Learning16 May 2025
- 2025CSE-SFP: Enabling Unsupervised Sentence Representation Learning via a Single Forward Pass1 May 2025
- 2025MoLEx: Mixture of Layer Experts for Finetuning with Sparse Upcycling14 Mar 2025
- 2025Domain Adaptation for Japanese Sentence Embeddings with Contrastive Learning based on Synthetic Sentence Generation12 Mar 2025
- 2025Continuous Adversarial Text Representation Learning for Affective Recognition28 Feb 2025
- 2025Interpretable Text Embeddings and Text Similarity Explanation: A Survey20 Feb 2025
- 2025Refining Sentence Embedding Model through Ranking Sentences Generation with Large Language Models19 Feb 2025
- 2025The Shrinking Landscape of Linguistic Diversity in the Age of Large Language Models16 Feb 2025
- 2024BEExformer: A Fast Inferencing Binarized Transformer with Early Exits6 Dec 2024
- 2024ChemTEB: Chemical Text Embedding Benchmark, an Overview of Embedding Models Performance & Efficiency on a Specific Domain30 Nov 2024
- 2024HNCSE: Advancing Sentence Embeddings via Hybrid Contrastive Learning with Hard Negatives19 Nov 2024
- 2024Eguard: Defending LLM Embeddings Against Inversion Attacks via Text Mutual Information Optimization6 Nov 2024
- 2024An Empirical Analysis of Speech Self-Supervised Learning at Multiple Resolutions31 Oct 2024
- 2024GenEOL: Harnessing the Generative Power of LLMs for Training-Free Sentence Embeddings18 Oct 2024
- 2024Pixology: Probing the Linguistic and Visual Capabilities of Pixel-based Language Models15 Oct 2024
- 2024GLOV: Guided Large Language Models as Implicit Optimizers for Vision Language Models8 Oct 2024
- 2024Instruction Embedding: Latent Representations of Instructions Towards Task Identification29 Sep 2024
- 2024Enhancing Unsupervised Sentence Embeddings via Knowledge-Driven Data Augmentation and Gaussian-Decayed Contrastive Learning19 Sep 2024
- 2024MEXMA: Token-level objectives improve sentence representations19 Sep 2024
- 2024Distilling Monolingual and Crosslingual Word-in-Context Representations13 Sep 2024
- 2024The Russian-focused embedders' exploration: ruMTEB benchmark and Russian embedding model design22 Aug 2024
- 2024Improving embedding with contrastive fine-tuning on small datasets with expert-augmented scores19 Aug 2024
- 2024Towards Flexible Evaluation for Generative Visual Question Answering1 Aug 2024
- 2024Word Embedding Dimension Reduction via Weakly-Supervised Feature Selection17 Jul 2024
50 newest of 294 All 294 (JSON)
- 2024GLOV: Guided Large Language Models as Implicit Optimizers for Vision Language Modelscited by 6
- 2024Towards Flexible Evaluation for Generative Visual Question Answeringcited by 2
- 2025Instruction-Following Evaluation of Large Vision-Language Modelscited by 1
- 2026Cross-Modal Masked Compositional Concept Modeling for Enhancing Visio-Linguistic Compositionalitycited by 0
- 2026Cross-Modal Masked Compositional Concept Modeling for Enhancing Visio-Linguistic Compositionality11 Jun 2026
- 2025Instruction-Following Evaluation of Large Vision-Language Models29 Dec 2025
- 2024GLOV: Guided Large Language Models as Implicit Optimizers for Vision Language Models8 Oct 2024
- 2024Towards Flexible Evaluation for Generative Visual Question Answering1 Aug 2024
- 2021SimCSE: Simple Contrastive Learning of Sentence Embeddingscited by 2,168
- 2022Text Embeddings by Weakly-Supervised Contrastive Pre-trainingcited by 1,216
- 2023AGIEval: A Human-Centric Benchmark for Evaluating Foundation Modelscited by 701
- 2022MTEB: Massive Text Embedding Benchmarkcited by 623
- 2023C-Pack: Packed Resources For General Chinese Embeddingscited by 556
- 2021Sentence-T5: Scalable Sentence Encoders from Pre-trained Text-to-Text Modelscited by 462
- 2021Dynabench: Rethinking Benchmarking in NLPcited by 447
- 2023Larger language models do in-context learning differentlycited by 386
- 2022Text and Code Embeddings by Contrastive Pre-Trainingcited by 358
- 2021AI and the Everything in the Whole Wide World Benchmarkcited by 340
- 2022One Embedder, Any Task: Instruction-Finetuned Text Embeddingscited by 329
- 2021LexGLUE: A Benchmark Dataset for Legal Language Understanding in Englishcited by 302
12 most cited of 280 All 280 (JSON)
- 2026AISA: AI Safety Assistant Framework for Continuous Improvement of Highway Construction17 Aug 2026
- 2026MTEB-BR: A Text Embedding Benchmark for Brazilian Portuguese6 Jul 2026
- 2026TALAS: Teacher-Anchored Layer Alignment with Adaptive Sharpness-Aware Minimization for Embedding Distillation20 Jun 2026
- 2026MIC: Maximizing Informational Capacity in Adaptive Representations via Isotropic Subspace Alignment28 May 2026
- 2026Hubness, Not Anisotropy, Drives Cross-Lingual Retrieval Asymmetry in Multilingual Embedding Models26 May 2026
- 2026Universal Activation Verbalizer: A Unified Framework for Cross-Model Activation Explanation25 May 2026
- 2026MIPIC: Matryoshka Representation Learning via Self-Distilled Intra-Relational and Progressive Information Chaining27 Apr 2026
- 2026Predict and Reconstruct: Joint Objectives for Self-Supervised Language Representation Learning16 Apr 2026
- 2026Computational Lesions in Multilingual Language Models Separate Shared and Language-specific Brain Alignment12 Apr 2026
- 2026Understanding LLM Performance Degradation in Multi-Instance Processing: The Roles of Instance Count and Context Length23 Mar 2026
- 2026LMEB: Long-horizon Memory Embedding Benchmark13 Mar 2026
- 2026Bipartite Mode Matching for Vision Training Set Search from a Hierarchical Data Server14 Jan 2026
12 newest of 280 All 280 (JSON)