mlsanityData to 14 Oct 2026

Improve Dense Passage Retrieval with Entailment Tuning

arXiv 2410.15801 ↗First posted 21 Oct 2024cs.CL cs.IRcited by 7

On the map: Retrieval-augmented generation → Generation and context integration

Abstract (arXiv)

Retrieval module can be plugged into many downstream NLP tasks to improve their performance, such as open-domain question answering and retrieval-augmented generation. The key to a retrieval system is to calculate relevance scores to query and passage pairs. However, the definition of relevance is often ambiguous. We observed that a major class of relevance aligns with the concept of entailment in NLI tasks. Based on this observation, we designed a method called entailment tuning to improve the embedding of dense retrievers. Specifically, we unify the form of retrieval data and NLI data using existence claim as a bridge. Then, we train retrievers to predict the claims entailed in a passage with a variant task of masked prediction. Our method can be efficiently plugged into current dense retrieval methods, and experiments show the effectiveness of our method.

Key facts. Cited by 7 papers in the map (data through 14 Oct 2026). Placed in Generation and context integration, one of 15 directions in Retrieval-augmented generation. Builds on 36 of its 46 references that are in the map. First posted on arXiv 21 Oct 2024.

Builds on

36 of 46 references.

Closest ancestors · 3

Same direction, newest first

  1. 2024Reliable, Adaptable, and Attributable Language Models with Retrieval5 Mar
  2. 2020Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks22 May
  3. 2020REALM: Retrieval-Augmented Language Model Pre-Training10 Feb

Other references · 33

  1. 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,599
  2. 2019RoBERTa: A Robustly Optimized BERT Pretraining Approachcited by 13,372
  3. 2019Sentence-BERT: Sentence Embeddings using Siamese BERT-Networkscited by 9,170
  4. 2020Dense Passage Retrieval for Open-Domain Question Answeringcited by 3,824
  5. 2017A Broad-Coverage Challenge Corpus for Sentence Understanding through Inferencecited by 2,682
  6. 2017Billion-scale similarity search with GPUscited by 2,497
  7. 2020DeBERTa: Decoding-enhanced BERT with Disentangled Attentioncited by 2,181
  8. 2021SimCSE: Simple Contrastive Learning of Sentence Embeddingscited by 2,173
  9. 2015A large annotated corpus for learning natural language inferencecited by 1,847
  10. 2021Unsupervised Dense Information Retrieval with Contrastive Learningcited by 1,216
  11. 2020Leveraging Passage Retrieval with Generative Models for Open Domain Question Answeringcited by 1,210
  12. 2020ColBERT: Efficient and Effective Passage Search via Contextualized Late Interaction over BERTcited by 1,178
21 more
  1. 2014Convolutional Neural Networks for Sentence Classificationcited by 1,127
  2. 2017Reading Wikipedia to Answer Open-Domain Questionscited by 777
  3. 2019Latent Retrieval for Weakly Supervised Open Domain Question Answeringcited by 695
  4. 2019ELI5: Long Form Question Answeringcited by 575
  5. 2022Precise Zero-Shot Dense Retrieval without Relevance Labelscited by 517
  6. 2021Sentence-T5: Scalable Sentence Encoders from Pre-trained Text-to-Text Modelscited by 463
  7. 2017Supervised Learning of Universal Sentence Representations from Natural Language Inference Datacited by 432
  8. 2020RocketQA: An Optimized Training Approach to Dense Passage Retrieval for Open-Domain Question Answeringcited by 413
  9. 2024LLM2Vec: Large Language Models Are Secretly Powerful Text Encoderscited by 404
  10. 2022One Embedder, Any Task: Instruction-Finetuned Text Embeddingscited by 333
  11. 2022ASQA: Factoid Questions Meet Long-Form Answerscited by 234
  12. 2021Hurdles to Progress in Long-form Question Answeringcited by 167
  13. 2019Poly-encoders: Transformer Architectures and Pre-training Strategies for Fast and Accurate Multi-sentence Scoringcited by 165
  14. 2021Condenser: a Pre-training Architecture for Dense Retrievalcited by 162
  15. 2022Should You Mask 15% in Masked Language Modeling?cited by 118
  16. 2022RetroMAE: Pre-Training Retrieval-oriented Language Models Via Masked Auto-Encodercited by 118
  17. 2018The Fact Extraction and VERification (FEVER) Shared Taskcited by 108
  18. 2022Task-aware Retrieval with Instructionscited by 107
  19. 2022Neighborhood Contrastive Learning for Scientific Document Representations with Citation Embeddingscited by 75
  20. 2024Bridging the Preference Gap between Retrievers and LLMscited by 69
  21. 2022LawngNLI: A Long-Premise Benchmark for In-Domain Generalization from Short to Long Contexts and for Implication-Based Retrievalcited by 6

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 5; 2026: 2.

7 papers.

  1. 2025SePer: Measure Retrieval Utility Through The Lens Of Semantic Perplexity Reductioncited by 24
  2. 2025Mark Your LLM: Detecting the Misuse of Open-Source Large Language Models via Watermarkingcited by 19
  3. 2025Training a Utility-based Retriever Through Shared Context Attribution for Retrieval-Augmented Language Modelscited by 13
  4. 2026LLM-Oriented Information Retrieval: A Denoising-First Perspectivecited by 1
  5. 2025SEAL: Structure and Element Aware Learning to Improve Long Structured Document Retrievalcited by 0
  6. 2025Rationale-Augmented Retrieval with Constrained LLM Re-Ranking for Task Discoverycited by 0
  7. 2026Automated Regulatory Compliance Question Answering in Financial Services with Domain-Adapted Retrieval-Augmented Generationcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2021PAIR: Leveraging Passage-Centric Similarity Relation for Improving Dense Passage Retrieval
  2. 2021Phrase Retrieval Learns Passage Retrieval, Too
  3. 2023Dense X Retrieval: What Retrieval Granularity Should We Use?
  4. 2023Query Encoder Distillation via Embedding Alignment is a Strong Baseline Method to Boost Dense Retriever Online Efficiency
  5. 2026Xetrieval: Mechanistically Explaining Dense Retrieval
  6. 2023Making Retrieval-Augmented Language Models Robust to Irrelevant Context
  7. 2024W-RAG: Weakly Supervised Dense Retrieval in RAG for Open-domain Question Answering
  8. 2022LED: Lexicon-Enlightened Dense Retriever for Large-Scale Retrieval