mlsanityData to 7 Oct 2026

Meta-Task Prompting Elicits Embeddings from Large Language Models

arXiv 2402.18458 ↗First posted 28 Feb 2024cs.CLcited by 15

Not on the map. Why?

Abstract (arXiv)

We introduce a new unsupervised text embedding method, Meta-Task Prompting with Explicit One-Word Limitation (MetaEOL), for generating high-quality sentence embeddings from Large Language Models (LLMs) without the need for model fine-tuning. Leveraging meta-task prompting, MetaEOL guides LLMs to produce embeddings through a series of carefully designed prompts that address multiple representational aspects. Our comprehensive experiments demonstrate that embeddings averaged from various meta-tasks are versatile embeddings that yield competitive performance on Semantic Textual Similarity (STS) benchmarks and excel in downstream tasks, surpassing contrastive-trained models. Our findings suggest a new scaling law, offering a versatile and resource-efficient approach for embedding generation across diverse scenarios.

Key facts. Cited by 15 papers in the map (data through 7 Oct 2026). Builds on 30 of its 58 references that are in the map. First posted on arXiv 28 Feb 2024.

Builds on

30 of 58 references.

References · 30

  1. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  2. 2020Language Models are Few-Shot Learnerscited by 32,199
  3. 2023LLaMA: Open and Efficient Foundation Language Modelscited by 13,958
  4. 2019RoBERTa: A Robustly Optimized BERT Pretraining Approachcited by 13,266
  5. 2019Sentence-BERT: Sentence Embeddings using Siamese BERT-Networkscited by 9,049
  6. 2021Finetuned Language Models Are Zero-Shot Learnerscited by 3,747
  7. 2022OPT: Open Pre-trained Transformer Language Modelscited by 3,464
  8. 2021SimCSE: Simple Contrastive Learning of Sentence Embeddingscited by 2,168
  9. 2021Unsupervised Dense Information Retrieval with Contrastive Learningcited by 1,203
  10. 2017SemEval-2017 Task 1: Semantic Textual Similarity - Multilingual and Cross-lingual Focused Evaluationcited by 998
  11. 2022Super-NaturalInstructions: Generalization via Declarative Instructions on 1600+ NLP Taskscited by 858
  12. 2019EDA: Easy Data Augmentation Techniques for Boosting Performance on Text Classification Taskscited by 800
18 more
  1. 2022A Survey on In-context Learningcited by 717
  2. 2021Cross-Task Generalization via Natural Language Crowdsourcing Instructionscited by 696
  3. 2023Is ChatGPT a General-Purpose Natural Language Processing Task Solver?cited by 486
  4. 2021Sentence-T5: Scalable Sentence Encoders from Pre-trained Text-to-Text Modelscited by 462
  5. 2021Large Dual Encoders Are Generalizable Retrieverscited by 454
  6. 2022One Embedder, Any Task: Instruction-Finetuned Text Embeddingscited by 329
  7. 2018SentEval: An Evaluation Toolkit for Universal Sentence Representationscited by 294
  8. 2023Fine-Tuning LLaMA for Multi-Stage Text Retrievalcited by 271
  9. 2023Can ChatGPT Understand Too? A Comparative Study on ChatGPT and Fine-tuned BERTcited by 155
  10. 2022DiffCSE: Difference-based Contrastive Learning for Sentence Embeddingscited by 131
  11. 2022Task-aware Retrieval with Instructionscited by 107
  12. 2016Learning to Match Using Local and Distributed Representations of Text for Web Searchcited by 68
  13. 2022HyperPrompt: Prompt-based Task-Conditioning of Transformerscited by 67
  14. 2023BeLLM: Backward Dependency Enhanced Large Language Model for Sentence Embeddingscited by 32
  15. 2022PCL: Peer-Contrastive Learning with Diverse Augmentations for Unsupervised Sentence Embeddingscited by 29
  16. 2023Improving Contrastive Learning of Sentence Embeddings from AI Feedbackcited by 29
  17. 2023Contrastive Learning of Sentence Embeddings from Scratchcited by 24
  18. 2023Meaning Representations from Trajectories in Autoregressive Modelscited by 18

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 5; 2025: 3; 2026: 7.

15 papers.

Sort:

By area:

  1. 2024LLM2Vec: Large Language Models Are Secretly Powerful Text Encoderscited by 396
  2. 2024Your Mixture-of-Experts LLM Is Secretly an Embedding Model For Freecited by 37
  3. 2024Simple Techniques for Enhancing Sentence Embeddings in Generative Language Modelscited by 26
  4. 2024FLAME: Frozen Large Language Models Enable Data-Efficient Language-Image Pre-trainingcited by 12
  5. 2024Enhancing Few-Shot Vision-Language Classification with Large Multimodal Model Featurescited by 11
  6. 2025FreeRet: MLLMs as Training-Free Retrieverscited by 7
  7. 2026LLM2Vec-Gen: Generative Embeddings from Large Language Modelscited by 2
  8. 2025Context-Adaptive Multi-Prompt Embedding with Large Language Models for Vision-Language Alignmentcited by 1
  9. 2026LLM-based Embeddings: Attention Values Encode Sentence Semantics Better Than Hidden Statescited by 1
  10. 2026mEOL: Training-Free Instruction-Guided Multimodal Embedder for Vector Graphics and Image Retrievalcited by 1
  11. 2026Your UnEmbedding Matrix is Secretly a Feature Lens for Text Embeddingscited by 1
  12. 2025KunlunBaize: LLM with Multi-Scale Convolution and Multi-Token Prediction Under TransformerX Frameworkcited by 0
3 more
  1. 2026Lens: Bringing the Right Semantic Perspective into Focus for Training-Free Multimodal Representation Learningcited by 0
  2. 2026Coherence-Aware Distributional Evaluation of Open-Ended Text Generationcited by 0
  3. 2026Selecting What Matters: Semantic Compression-Guided Selective Pooling for Long-Context Embeddingscited by 0