mlsanityData to 7 Oct 2026

GeAR: Generation Augmented Retrieval

arXiv 2501.02772 ↗First posted 6 Jan 2025cs.IR cs.CLcited by 0

Not on the map. Why?

Abstract (arXiv)

Document retrieval techniques are essential for developing large-scale information systems. The common approach involves using a bi-encoder to compute the semantic similarity between a query and documents. However, the scalar similarity often fail to reflect enough information, hindering the interpretation of retrieval results. In addition, this process primarily focuses on global semantics, overlooking the finer-grained semantic relationships between the query and the document's content. In this paper, we introduce a novel method, Generation Augmented Retrieval (GeAR), which not only improves the global document-query similarity through contrastive learning, but also integrates well-designed fusion and decoding modules. This enables GeAR to generate relevant context within the documents based on a given query, facilitating learning to retrieve local fine-grained information. Furthermore, when used as a retriever, GeAR does not incur any additional computational cost over bi-encoders. GeAR exhibits competitive retrieval performance across diverse scenarios and tasks. Moreover, qualitative analysis and the results generated by GeAR provide novel insights into the interpretation of retrieval results. The code, data, and models will be released at hrefhttps://github.com/microsoft/LMOpshttps://github.com/microsoft/LMOps.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 37 of its 54 references that are in the map. First posted on arXiv 6 Jan 2025.

Builds on

37 of 54 references.

References · 37

  1. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  2. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
  3. 2024The Llama 3 Herd of Modelscited by 14,192
  4. 2020Retrieval-Augmented Generation for Knowledge-Intensive NLP Taskscited by 9,160
  5. 2019Sentence-BERT: Sentence Embeddings using Siamese BERT-Networkscited by 9,049
  6. 2023Efficient Memory Management for Large Language Model Serving with PagedAttentioncited by 5,767
  7. 2016SQuAD: 100,000+ Questions for Machine Comprehension of Textcited by 4,115
  8. 2020Dense Passage Retrieval for Open-Domain Question Answeringcited by 3,779
  9. 2013Distributed Representations of Words and Phrases and their Compositionalitycited by 3,462
  10. 2023Lost in the Middle: How Language Models Use Long Contextscited by 3,383
  11. 2017TriviaQA: A Large Scale Distantly Supervised Challenge Dataset for Reading Comprehensioncited by 2,895
  12. 2017Billion-scale similarity search with GPUscited by 2,477
25 more
  1. 2023Retrieval-Augmented Generation for Large Language Models: A Surveycited by 2,185
  2. 2021SimCSE: Simple Contrastive Learning of Sentence Embeddingscited by 2,168
  3. 2021Align before Fuse: Vision and Language Representation Learning with Momentum Distillationcited by 1,612
  4. 2022Text Embeddings by Weakly-Supervised Contrastive Pre-trainingcited by 1,216
  5. 2020ColBERT: Efficient and Effective Passage Search via Contextualized Late Interaction over BERTcited by 1,160
  6. 2020MPNet: Masked and Permuted Pre-training for Language Understandingcited by 973
  7. 2020Approximate Nearest Neighbor Negative Contrastive Learning for Dense Text Retrievalcited by 846
  8. 2018Multi-Task Learning as Multi-Objective Optimizationcited by 809
  9. 2017Reading Wikipedia to Answer Open-Domain Questionscited by 777
  10. 2023Towards General Text Embeddings with Multi-stage Contrastive Learningcited by 705
  11. 2023Enabling Large Language Models to Generate Text with Citationscited by 584
  12. 2023C-Pack: Packed Resources For General Chinese Embeddingscited by 556
  13. 2016Bidirectional Attention Flow for Machine Comprehensioncited by 278
  14. 2021Efficiently Teaching an Effective Dense Retriever with Balanced Topic Aware Samplingcited by 238
  15. 2024Generative Representational Instruction Tuningcited by 227
  16. 2021PAQ: 65 Million Probably-Asked Questions and What You Can Do With Themcited by 198
  17. 2019BERT Post-Training for Review Reading Comprehension and Aspect-based Sentiment Analysiscited by 176
  18. 2024Leave No Document Behind: Benchmarking Long-Context LLMs with Extended Multi-Doc QAcited by 121
  19. 2024Make Your LLM Fully Utilize the Contextcited by 118
  20. 2024LongCite: Enabling LLMs to Generate Fine-grained Citations in Long-context QAcited by 78
  21. 2020Relevance-guided Supervision for OpenQA with ColBERTcited by 71
  22. 2024Late Chunking: Contextual Chunk Embeddings Using Long-Context Embedding Modelscited by 70
  23. 2016Machine Comprehension Using Match-LSTM and Answer Pointercited by 47
  24. 2024LongEmbed: Extending Embedding Models for Long Context Retrievalcited by 47
  25. 2021Dense Hierarchical Retrieval for Open-Domain Question Answeringcited by 33

Built on by

Not cited yet.