mlsanityData to 7 Oct 2026

Answer is All You Need: Instruction-following Text Embedding via Answering the Question

arXiv 2402.09642 ↗First posted 15 Feb 2024cs.CLcited by 26

Not on the map. Why?

Abstract (arXiv)

This work aims to build a text embedder that can capture characteristics of texts specified by user instructions. Despite its tremendous potential to deploy user-oriented embeddings, none of previous approaches provides a concrete solution for it. This paper offers a new viewpoint, which treats the instruction as a question about the input text and encodes the expected answers to obtain the representation accordingly. Intuitively, texts with the same (implicit) semantics would share similar answers following the instruction, thus leading to more similar embeddings. Specifically, we propose InBedder that instantiates this embed-via-answering idea by only fine-tuning language models on abstractive question answering tasks. InBedder demonstrates significantly improved instruction-following capabilities according to our proposed instruction awareness tests and instruction robustness tests, when applied to both large language models (LLMs) (e.g., llama-2-7b) and smaller encoder-based LMs (e.g., roberta-large). Additionally, our qualitative analysis of clustering outcomes, achieved by applying different instructions to the same corpus, demonstrates a high degree of interpretability.

Key facts. Cited by 26 papers in the map (data through 7 Oct 2026). Builds on 23 of its 37 references that are in the map. First posted on arXiv 15 Feb 2024.

Builds on

23 of 37 references.

References · 23

  1. 2022Training language models to follow instructions with human feedbackcited by 15,712
  2. 2023LLaMA: Open and Efficient Foundation Language Modelscited by 13,958
  3. 2023Llama 2: Open Foundation and Fine-Tuned Chat Modelscited by 12,865
  4. 2019Sentence-BERT: Sentence Embeddings using Siamese BERT-Networkscited by 9,049
  5. 2020Dense Passage Retrieval for Open-Domain Question Answeringcited by 3,779
  6. 2022Scaling Instruction-Finetuned Language Modelscited by 2,997
  7. 2015Character-level Convolutional Networks for Text Classificationcited by 2,293
  8. 2019HuggingFace's Transformers: State-of-the-art Natural Language Processingcited by 2,201
  9. 2021SimCSE: Simple Contrastive Learning of Sentence Embeddingscited by 2,168
  10. 2023Representation Engineering: A Top-Down Approach to AI Transparencycited by 1,239
  11. 2022Text Embeddings by Weakly-Supervised Contrastive Pre-trainingcited by 1,216
  12. 2017SemEval-2017 Task 1: Semantic Textual Similarity - Multilingual and Cross-lingual Focused Evaluationcited by 998
11 more
  1. 2021Sentence-T5: Scalable Sentence Encoders from Pre-trained Text-to-Text Modelscited by 462
  2. 2021Large Dual Encoders Are Generalizable Retrieverscited by 454
  3. 2019Multi-News: a Large-Scale Multi-Document Summarization Dataset and Abstractive Hierarchical Modelcited by 431
  4. 2020Efficient Intent Detection with Dual Sentence Encoderscited by 419
  5. 2022One Embedder, Any Task: Instruction-Finetuned Text Embeddingscited by 329
  6. 2018SentEval: An Evaluation Toolkit for Universal Sentence Representationscited by 294
  7. 2020Multi-XScience: A Large-scale Dataset for Extreme Multi-document Summarization of Scientific Articlescited by 96
  8. 2023ClusterLLM: Large Language Models as a Guide for Text Clusteringcited by 69
  9. 2023Goal-Driven Explainable Clustering via Language Descriptionscited by 58
  10. 2023Controllable Data Augmentation for Few-Shot Text Mining with Chain-of-Thought Attribute Manipulationcited by 9
  11. 2018Learning and Evaluating Sparse Interpretable Sentence Embeddingscited by 5

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 4; 2025: 9; 2026: 13.

26 papers.

Sort:

By area:

  1. 2024Speculative RAG: Enhancing Retrieval Augmented Generation through Draftingcited by 76
  2. 2025Retrieval-Augmented Generation with Conflicting Evidencecited by 60
  3. 2024Crafting Interpretable Embeddings by Asking LLMs Questionscited by 11
  4. 2025Position: Text Embeddings Should Capture Implicit Semantics, Not Just Surface Meaningcited by 10
  5. 2025Let LLMs Speak Embedding Languages: Generative Text Embeddings via Iterative Contrastive Refinementcited by 9
  6. 2026Unlocking Multimodal Document Intelligence: From Current Triumphs to Future Frontiers of Visual Document Retrievalcited by 8
  7. 2026LaSER: Internalizing Explicit Reasoning into Latent Space for Dense Retrievalcited by 8
  8. 2025Don't Reinvent the Wheel: Efficient Instruction-Following Text Embedding based on Guided Space Transformationcited by 6
  9. 2025What Makes a Good Natural Language Prompt?cited by 6
  10. 2025PRISM: A Framework for Producing Interpretable Political Bias Embeddings with Political-Aware Cross-Encodercited by 5
  11. 2026CSRv2: Unlocking Ultra-Sparse Embeddingscited by 5
  12. 2025Panoramic Interests: Stylistic-Content Aware Personalized Headline Generationcited by 4
14 more
  1. 2024CodeRefine: A Pipeline for Enhancing LLM-Generated Code Implementations of Research Paperscited by 3
  2. 2025On The Role of Pretrained Language Models in General-Purpose Text Embeddings: A Surveycited by 3
  3. 2026LLM2Vec-Gen: Generative Embeddings from Large Language Modelscited by 2
  4. 2024Correlation and Navigation in the Vocabulary Key Representation Space of Language Modelscited by 1
  5. 2025Instruction-aware User Embedding via Synergistic Language and Representation Modelingcited by 1
  6. 2026Query as Anchor: Scenario-Adaptive User Representation via Large Language Modelcited by 1
  7. 2026Larch: Learned Query Optimization for Semantic Predicatescited by 1
  8. 2026One Instruction Does Not Fit All: How Well Do Embeddings Align Personas and Instructions in Low-Resource Indian Languages?cited by 0
  9. 2026Cross-lingual Matryoshka Representation Learning across Speech and Textcited by 0
  10. 2026No More K-means: Single-Stage Sparse Coding for Efficient Multi-Vector Retrievalcited by 0
  11. 2026Bridging Auxiliary Constraints to Resolve Instruction Following in Large Reasoning Modelscited by 0
  12. 2026Retrieval Grounding Latent Reasoning for Dense Retrievalcited by 0
  13. 2026CG-Probes: Recovering Guardrail Directions from Patient Query Embeddingscited by 0
  14. 2026Focusing Condition: Inference-Time Self-Contrastive Steering Elicits Better Conditional Text Embeddings in LLMscited by 0