# Answer is All You Need: Instruction-following Text Embedding via Answering the Question

arXiv: https://arxiv.org/abs/2402.09642 · HTML: https://mlsanity.com/paper/2402.09642/

## Key facts

- Cited by 27 papers in the map (data through 14 Oct 2026).
- Builds on 23 of its 37 references that are in the map.
- First posted on arXiv 15 Feb 2024.

## Builds on (most cited first, 23 of 23)

- [Training language models to follow instructions with human feedback](https://mlsanity.com/paper/2203.02155/) (2022)
- [LLaMA: Open and Efficient Foundation Language Models](https://mlsanity.com/paper/2302.13971/) (2023)
- [Llama 2: Open Foundation and Fine-Tuned Chat Models](https://mlsanity.com/paper/2307.09288/) (2023)
- [Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks](https://mlsanity.com/paper/1908.10084/) (2019)
- [Dense Passage Retrieval for Open-Domain Question Answering](https://mlsanity.com/paper/2004.04906/) (2020)
- [Scaling Instruction-Finetuned Language Models](https://mlsanity.com/paper/2210.11416/) (2022)
- [Character-level Convolutional Networks for Text Classification](https://mlsanity.com/paper/1509.01626/) (2015)
- [HuggingFace's Transformers: State-of-the-art Natural Language Processing](https://mlsanity.com/paper/1910.03771/) (2019)
- [SimCSE: Simple Contrastive Learning of Sentence Embeddings](https://mlsanity.com/paper/2104.08821/) (2021)
- [Representation Engineering: A Top-Down Approach to AI Transparency](https://mlsanity.com/paper/2310.01405/) (2023)
- [Text Embeddings by Weakly-Supervised Contrastive Pre-training](https://mlsanity.com/paper/2212.03533/) (2022)
- [SemEval-2017 Task 1: Semantic Textual Similarity - Multilingual and Cross-lingual Focused Evaluation](https://mlsanity.com/paper/1708.00055/) (2017)
- [Sentence-T5: Scalable Sentence Encoders from Pre-trained Text-to-Text Models](https://mlsanity.com/paper/2108.08877/) (2021)
- [Large Dual Encoders Are Generalizable Retrievers](https://mlsanity.com/paper/2112.07899/) (2021)
- [Efficient Intent Detection with Dual Sentence Encoders](https://mlsanity.com/paper/2003.04807/) (2020)
- [Multi-News: a Large-Scale Multi-Document Summarization Dataset and Abstractive Hierarchical Model](https://mlsanity.com/paper/1906.01749/) (2019)
- [One Embedder, Any Task: Instruction-Finetuned Text Embeddings](https://mlsanity.com/paper/2212.09741/) (2022)
- [SentEval: An Evaluation Toolkit for Universal Sentence Representations](https://mlsanity.com/paper/1803.05449/) (2018)
- [Multi-XScience: A Large-scale Dataset for Extreme Multi-document Summarization of Scientific Articles](https://mlsanity.com/paper/2010.14235/) (2020)
- [ClusterLLM: Large Language Models as a Guide for Text Clustering](https://mlsanity.com/paper/2305.14871/) (2023)
- [Goal-Driven Explainable Clustering via Language Descriptions](https://mlsanity.com/paper/2305.13749/) (2023)
- [Controllable Data Augmentation for Few-Shot Text Mining with Chain-of-Thought Attribute Manipulation](https://mlsanity.com/paper/2307.07099/) (2023)
- [Learning and Evaluating Sparse Interpretable Sentence Embeddings](https://mlsanity.com/paper/1809.08621/) (2018)

## Built on by (most cited first, 27 of 27)

- [Speculative RAG: Enhancing Retrieval Augmented Generation through Drafting](https://mlsanity.com/paper/2407.08223/) (2024)
- [Retrieval-Augmented Generation with Conflicting Evidence](https://mlsanity.com/paper/2504.13079/) (2025)
- [Crafting Interpretable Embeddings by Asking LLMs Questions](https://mlsanity.com/paper/2405.16714/) (2024)
- [Position: Text Embeddings Should Capture Implicit Semantics, Not Just Surface Meaning](https://mlsanity.com/paper/2506.08354/) (2025)
- [Let LLMs Speak Embedding Languages: Generative Text Embeddings via Iterative Contrastive Refinement](https://mlsanity.com/paper/2509.24291/) (2025)
- [LaSER: Internalizing Explicit Reasoning into Latent Space for Dense Retrieval](https://mlsanity.com/paper/2603.01425/) (2026)
- [Unlocking Multimodal Document Intelligence: From Current Triumphs to Future Frontiers of Visual Document Retrieval](https://mlsanity.com/paper/2602.19961/) (2026)
- [Don't Reinvent the Wheel: Efficient Instruction-Following Text Embedding based on Guided Space Transformation](https://mlsanity.com/paper/2505.24754/) (2025)
- [What Makes a Good Natural Language Prompt?](https://mlsanity.com/paper/2506.06950/) (2025)
- [CSRv2: Unlocking Ultra-Sparse Embeddings](https://mlsanity.com/paper/2602.05735/) (2026)
- [PRISM: A Framework for Producing Interpretable Political Bias Embeddings with Political-Aware Cross-Encoder](https://mlsanity.com/paper/2505.24646/) (2025)
- [Panoramic Interests: Stylistic-Content Aware Personalized Headline Generation](https://mlsanity.com/paper/2501.11900/) (2025)
- [Larch: Learned Query Optimization for Semantic Predicates](https://mlsanity.com/paper/2606.07923/) (2026)
- [CodeRefine: A Pipeline for Enhancing LLM-Generated Code Implementations of Research Papers](https://mlsanity.com/paper/2408.13366/) (2024)
- [On The Role of Pretrained Language Models in General-Purpose Text Embeddings: A Survey](https://mlsanity.com/paper/2507.20783/) (2025)
- [LLM2Vec-Gen: Generative Embeddings from Large Language Models](https://mlsanity.com/paper/2603.10913/) (2026)
- [Correlation and Navigation in the Vocabulary Key Representation Space of Language Models](https://mlsanity.com/paper/2410.02284/) (2024)
- [Instruction-aware User Embedding via Synergistic Language and Representation Modeling](https://mlsanity.com/paper/2510.11016/) (2025)
- [Query as Anchor: Scenario-Adaptive User Representation via Large Language Model](https://mlsanity.com/paper/2602.14492/) (2026)
- [No More K-means: Single-Stage Sparse Coding for Efficient Multi-Vector Retrieval](https://mlsanity.com/paper/2605.30120/) (2026)
- [Focusing Condition: Inference-Time Self-Contrastive Steering Elicits Better Conditional Text Embeddings in LLMs](https://mlsanity.com/paper/2609.32684/) (2026)
- [One Instruction Does Not Fit All: How Well Do Embeddings Align Personas and Instructions in Low-Resource Indian Languages?](https://mlsanity.com/paper/2601.10205/) (2026)
- [Cross-lingual Matryoshka Representation Learning across Speech and Text](https://mlsanity.com/paper/2602.19991/) (2026)
- [Bridging Auxiliary Constraints to Resolve Instruction Following in Large Reasoning Models](https://mlsanity.com/paper/2606.03624/) (2026)
- [Retrieval Grounding Latent Reasoning for Dense Retrieval](https://mlsanity.com/paper/2608.14107/) (2026)
- [CG-Probes: Recovering Guardrail Directions from Patient Query Embeddings](https://mlsanity.com/paper/2609.31062/) (2026)
- [Lend Me Your Eyes: Instruction-Aware Text Embeddings via Attention Relay](https://mlsanity.com/paper/2610.05564/) (2026)

## Abstract (arXiv)

This work aims to build a text embedder that can capture characteristics of texts specified by user instructions. Despite its tremendous potential to deploy user-oriented embeddings, none of previous approaches provides a concrete solution for it. This paper offers a new viewpoint, which treats the instruction as a question about the input text and encodes the expected answers to obtain the representation accordingly. Intuitively, texts with the same (implicit) semantics would share similar answers following the instruction, thus leading to more similar embeddings. Specifically, we propose InBedder that instantiates this embed-via-answering idea by only fine-tuning language models on abstractive question answering tasks. InBedder demonstrates significantly improved instruction-following capabilities according to our proposed instruction awareness tests and instruction robustness tests, when applied to both large language models (LLMs) (e.g., llama-2-7b) and smaller encoder-based LMs (e.g., roberta-large). Additionally, our qualitative analysis of clustering outcomes, achieved by applying different instructions to the same corpus, demonstrates a high degree of interpretability.