mlsanityData to 7 Oct 2026

One Embedder, Any Task: Instruction-Finetuned Text Embeddings

arXiv 2212.09741 ↗First posted 19 Dec 2022cs.CLcited by 329

Not on the map. Why?

Cited by 27 in Retrieval-augmented generation, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

We introduce INSTRUCTOR, a new method for computing text embeddings given task instructions: every text input is embedded together with instructions explaining the use case (e.g., task and domain descriptions). Unlike encoders from prior work that are more specialized, INSTRUCTOR is a single embedder that can generate text embeddings tailored to different downstream tasks and domains, without any further training. We first annotate instructions for 330 diverse tasks and train INSTRUCTOR on this multitask mixture with a contrastive loss. We evaluate INSTRUCTOR on 70 embedding evaluation tasks (66 of which are unseen during training), ranging from classification and information retrieval to semantic textual similarity and text generation evaluation. INSTRUCTOR, while having an order of magnitude fewer parameters than the previous best model, achieves state-of-the-art performance, with an average improvement of 3.4% compared to the previous best results on the 70 diverse datasets. Our analysis suggests that INSTRUCTOR is robust to changes in instructions, and that instruction finetuning mitigates the challenge of training a single model on diverse datasets. Our model, code, and data are available at https://instructor-embedding.github.io.

Key facts. Cited by 329 papers in the map (data through 7 Oct 2026). Builds on 63 of its 100 references that are in the map. First posted on arXiv 19 Dec 2022.

Builds on

63 of 100 references.

References · 63

  1. 2014Microsoft COCO: Common Objects in Contextcited by 17,498
  2. 2022Training language models to follow instructions with human feedbackcited by 15,712
  3. 2019Sentence-BERT: Sentence Embeddings using Siamese BERT-Networkscited by 9,049
  4. 2019BERTScore: Evaluating Text Generation with BERTcited by 5,462
  5. 2016SQuAD: 100,000+ Questions for Machine Comprehension of Textcited by 4,115
  6. 2020Dense Passage Retrieval for Open-Domain Question Answeringcited by 3,779
  7. 2021Finetuned Language Models Are Zero-Shot Learnerscited by 3,747
  8. 2018HotpotQA: A Dataset for Diverse, Explainable Multi-hop Question Answeringcited by 3,686
  9. 2017TriviaQA: A Large Scale Distantly Supervised Challenge Dataset for Reading Comprehensioncited by 2,895
  10. 2017A Broad-Coverage Challenge Corpus for Sentence Understanding through Inferencecited by 2,665
  11. 2021CLIPScore: A Reference-free Evaluation Metric for Image Captioningcited by 2,388
  12. 2015Character-level Convolutional Networks for Text Classificationcited by 2,293
38 more
  1. 2021Pre-train, Prompt, and Predict: A Systematic Survey of Prompting Methods in Natural Language Processingcited by 2,222
  2. 2021SimCSE: Simple Contrastive Learning of Sentence Embeddingscited by 2,168
  3. 2015A large annotated corpus for learning natural language inferencecited by 1,837
  4. 2016MS MARCO: A Human Generated MAchine Reading COmprehension Datasetcited by 1,591
  5. 2020Don't Stop Pretraining: Adapt Language Models to Domains and Taskscited by 1,548
  6. 2021Multitask Prompted Training Enables Zero-Shot Task Generalizationcited by 1,462
  7. 2018FEVER: a large-scale dataset for Fact Extraction and VERificationcited by 1,362
  8. 2018Don't Give Me the Details, Just the Summary! Topic-Aware Convolutional Neural Networks for Extreme Summarizationcited by 1,308
  9. 2021BEIR: A Heterogenous Benchmark for Zero-shot Evaluation of Information Retrieval Modelscited by 1,225
  10. 2022Text Embeddings by Weakly-Supervised Contrastive Pre-trainingcited by 1,216
  11. 2020Leveraging Passage Retrieval with Generative Models for Open Domain Question Answeringcited by 1,203
  12. 2021Unsupervised Dense Information Retrieval with Contrastive Learningcited by 1,203
  13. 2020BLEURT: Learning Robust Metrics for Text Generationcited by 1,196
  14. 2021What Makes Good In-Context Examples for GPT-3?cited by 1,184
  15. 2017SemEval-2017 Task 1: Semantic Textual Similarity - Multilingual and Cross-lingual Focused Evaluationcited by 998
  16. 2022Super-NaturalInstructions: Generalization via Declarative Instructions on 1600+ NLP Taskscited by 858
  17. 2021Cross-Task Generalization via Natural Language Crowdsourcing Instructionscited by 696
  18. 2020SummEval: Re-evaluating Summarization Evaluationcited by 666
  19. 2021Learning To Retrieve Prompts for In-Context Learningcited by 641
  20. 2019CodeSearchNet Challenge: Evaluating the State of Semantic Code Searchcited by 638
  21. 2022MTEB: Massive Text Embedding Benchmarkcited by 623
  22. 2020Contrastive Learning with Hard Negative Samplescited by 619
  23. 2020KILT: a Benchmark for Knowledge Intensive Language Taskscited by 571
  24. 2019ELI5: Long Form Question Answeringcited by 567
  25. 2018Wizard of Wikipedia: Knowledge-Powered Conversational agentscited by 530
  26. 2022MedMCQA : A Large-scale Multi-Subject Multi-Choice Dataset for Medical domain Question Answeringcited by 525
  27. 2016Ups and Downs: Modeling the Visual Evolution of Fashion Trends with One-Class Collaborative Filteringcited by 517
  28. 2021Sentence-T5: Scalable Sentence Encoders from Pre-trained Text-to-Text Modelscited by 462
  29. 2021Large Dual Encoders Are Generalizable Retrieverscited by 454
  30. 2021MetaICL: Learning to Learn In Contextcited by 450
  31. 2017Supervised Learning of Universal Sentence Representations from Natural Language Inference Datacited by 428
  32. 2020SPECTER: Document-level Representation Learning using Citation-informed Transformerscited by 426
  33. 2017Zero-Shot Relation Extraction via Reading Comprehensioncited by 425
  34. 2021Experts, Errors, and Context: A Large-Scale Study of Human Evaluation for Machine Translationcited by 333
  35. 2015Skip-Thought Vectorscited by 321
  36. 2018SentEval: An Evaluation Toolkit for Universal Sentence Representationscited by 294
  37. 2022Selective Annotation Makes Language Models Better Few-Shot Learnerscited by 242
  38. 2022SGPT: GPT Sentence Embeddings for Semantic Searchcited by 204

50 most cited of 63 All 63 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2022: 2; 2023: 54; 2024: 120; 2025: 94; 2026: 59.

329 papers.

Sort:

By area:

  1. 2025Qwen3 Embedding: Advancing Text Embedding and Reranking Through Foundation Modelscited by 1,175
  2. 2023REPLUG: Retrieval-Augmented Black-Box Language Modelscited by 786
  3. 2023Towards General Text Embeddings with Multi-stage Contrastive Learningcited by 705
  4. 2023C-Pack: Packed Resources For General Chinese Embeddingscited by 556
  5. 2022Precise Zero-Shot Dense Retrieval without Relevance Labelscited by 508
  6. 2023How Far Can Camels Go? Exploring the State of Instruction Tuning on Open Resourcescited by 405
  7. 2024LLM2Vec: Large Language Models Are Secretly Powerful Text Encoderscited by 396
  8. 2023Large Language Models for Information Retrieval: A Surveycited by 331
  9. 2023Improving Text Embeddings with Large Language Modelscited by 295
  10. 2023Evaluating Large Language Models at Evaluating Instruction Followingcited by 290
  11. 2024Nomic Embed: Training a Reproducible Long Context Text Embeddercited by 255
  12. 2024MultiHop-RAG: Benchmarking Retrieval-Augmented Generation for Multi-Hop Queriescited by 253
38 more
  1. 2023Magicoder: Empowering Code Generation with OSS-Instructcited by 251
  2. 2024Generative Representational Instruction Tuningcited by 227
  3. 2024VLM2Vec: Training Vision-Language Models for Massive Multimodal Embedding Taskscited by 222
  4. 2023AutoDroid: LLM-powered Task Automation in Androidcited by 178
  5. 2023UniIR: Training and Benchmarking Universal Multimodal Information Retrieverscited by 165
  6. 2024MM-Embed: Universal Multimodal Retrieval with Multimodal LLMscited by 159
  7. 2023Text2Reward: Reward Shaping with Language Models for Reinforcement Learningcited by 153
  8. 2024BRIGHT: A Realistic and Challenging Benchmark for Reasoning-Intensive Retrievalcited by 153
  9. 2023Unified Demonstration Retriever for In-Context Learningcited by 141
  10. 2024jina-embeddings-v3: Multilingual Embeddings With Task LoRAcited by 138
  11. 2023OpenAgents: An Open Platform for Language Agents in the Wildcited by 136
  12. 2025EmbeddingGemma: Powerful and Lightweight Text Representationscited by 128
  13. 2024MagicLens: Self-Supervised Image Retrieval with Open-Ended Instructionscited by 119
  14. 2023ChatEDA: A Large Language Model Powered Autonomous Agent for EDAcited by 118
  15. 2025VLM2Vec-V2: Advancing Multimodal Embedding for Videos, Images, and Visual Documentscited by 110
  16. 2024SaySelf: Teaching LLMs to Express Confidence with Self-Reflective Rationalescited by 102
  17. 2025On the Theoretical Limitations of Embedding-Based Retrievalcited by 98
  18. 2024CodeRAG-Bench: Can Retrieval Augment Code Generation?cited by 89
  19. 2024Large Concept Models: Language Modeling in a Sentence Representation Spacecited by 86
  20. 2024MegaPairs: Massive Data Synthesis For Universal Multimodal Retrievalcited by 84
  21. 2024Gecko: Versatile Text Embeddings Distilled from Large Language Modelscited by 81
  22. 2023Exploring the Benefits of Training Expert Language Models over Instruction Tuningcited by 80
  23. 2024Making Text Embedders Few-Shot Learnerscited by 78
  24. 2023In-context Pretraining: Language Modeling Beyond Document Boundariescited by 76
  25. 2023UPRISE: Universal Prompt Retrieval for Improving Zero-Shot Evaluationcited by 74
  26. 2024Reliable, Adaptable, and Attributable Language Models with Retrievalcited by 73
  27. 2023Large Language Models Enable Few-Shot Clusteringcited by 72
  28. 2023Lemur: Harmonizing Natural Language and Code for Language Agentscited by 71
  29. 2024Repetition Improves Language Model Embeddingscited by 70
  30. 2023ClusterLLM: Large Language Models as a Guide for Text Clusteringcited by 69
  31. 2024LoraRetriever: Input-Aware LoRA Retrieval and Composition for Mixed Tasks in the Wildcited by 68
  32. 2025Rank1: Test-Time Compute for Reranking in Information Retrievalcited by 68
  33. 2024FollowIR: Evaluating and Teaching Information Retrieval Models to Follow Instructionscited by 67
  34. 2023A Multi-Task Embedder For Retrieval Augmented LLMscited by 63
  35. 2024MobileAgentBench: An Efficient and User-Friendly Benchmark for Mobile LLM Agentscited by 63
  36. 2023Explaining black box text modules in natural language with language modelscited by 61
  37. 2025A Survey on (M)LLM-Based GUI Agentscited by 61
  38. 2023Automating Customer Service using LangChain: Building custom open-source GPT Chatbot for organizationscited by 59

50 most cited of 329 All 329 (JSON)