mlsanityData to 13 Oct 2026

Enhanced vectors for top-k document retrieval in Question Answering

arXiv 2210.10584 ↗First posted 8 Oct 2022cs.IR cs.CL cs.LGcited by 0

Not on the map. Why?

Abstract (arXiv)

Modern day applications, especially information retrieval webapps that involve "search" as their use cases are gradually moving towards "answering" modules. Conversational chatbots which have been proved to be more engaging to users, use Question Answering as their core. Since, precise answering is computationally expensive, several approaches have been developed to prefetch the most relevant documents/passages from the database that contain the answer. We propose a different approach that retrieves the evidence documents efficiently and accurately, making sure that the relevant document for a given user query is not missed. We do so by assigning each document (or passage in our case), a unique identifier and using them to create dense vectors which can be efficiently indexed. More precisely, we use the identifier to predict randomly sampled context window words of the relevant question corresponding to the passage along with the words of passage itself. This naturally embeds the passage identifier into the vector space in such a way that the embedding is closer to the question without compromising he information content. This approach enables efficient creation of real-time query vectors in ~4 milliseconds.

Key facts. No citing papers in the map yet (data through 13 Oct 2026). Builds on 32 of its 40 references that are in the map. First posted on arXiv 8 Oct 2022.

Builds on

32 of 40 references.

References · 32

  1. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,304
  2. 2015Distilling the Knowledge in a Neural Networkcited by 10,180
  3. 2020Retrieval-Augmented Generation for Knowledge-Intensive NLP Taskscited by 9,221
  4. 2016SQuAD: 100,000+ Questions for Machine Comprehension of Textcited by 4,133
  5. 2020Dense Passage Retrieval for Open-Domain Question Answeringcited by 3,797
  6. 2017Billion-scale similarity search with GPUscited by 2,490
  7. 2020REALM: Retrieval-Augmented Language Model Pre-Trainingcited by 2,131
  8. 2021BEIR: A Heterogenous Benchmark for Zero-shot Evaluation of Information Retrieval Modelscited by 1,237
  9. 2020ColBERT: Efficient and Effective Passage Search via Contextualized Late Interaction over BERTcited by 1,169
  10. 2017Reading Wikipedia to Answer Open-Domain Questionscited by 777
  11. 2019Latent Retrieval for Weakly Supervised Open Domain Question Answeringcited by 693
  12. 2014Distributed Representations of Sentences and Documentscited by 677
20 more
  1. 2019Passage Re-ranking with BERTcited by 638
  2. 2021ColBERTv2: Effective and Efficient Retrieval via Lightweight Late Interactioncited by 468
  3. 2021SPLADE: Sparse Lexical and Expansion Model for First Stage Rankingcited by 284
  4. 2020Sparse, Dense, and Attentional Representations for Text Retrievalcited by 214
  5. 2021COIL: Revisit Exact Lexical Match in Information Retrieval with Contextualized Inverted Listcited by 113
  6. 2017End-to-End Neural Ad-hoc Ranking with Kernel Poolingcited by 85
  7. 2019A Deep Look into Neural Ranking Models for Information Retrievalcited by 79
  8. 2020Relevance-guided Supervision for OpenQA with ColBERTcited by 71
  9. 2021A Few Brief Notes on DeepImpact, COIL, and a Conceptual Framework for Information Retrieval Techniquescited by 69
  10. 2021Learning Passage Impacts for Inverted Indexescited by 68
  11. 2019Context-Aware Sentence/Passage Term Importance Estimation For First Stage Retrievalcited by 65
  12. 2021GooAQ: Open Question Answering with Diverse Answer Typescited by 53
  13. 2021Efficient Passage Retrieval with Hashing for Open-domain Question Answeringcited by 53
  14. 2020Efficient Document Re-Ranking for Transformers by Precomputing Term Representationscited by 44
  15. 2021Jointly Optimizing Query Encoder and Product Quantization to Improve Retrieval Performancecited by 35
  16. 2020A Memory Efficient Baseline for Open Domain Question Answeringcited by 32
  17. 2020Is Retriever Merely an Approximator of Reader?cited by 30
  18. 2021Learning Discrete Representations via Constrained Clustering for Effective and Efficient Dense Retrievalcited by 27
  19. 2020Progressively Pretrained Dense Corpus Index for Open-Domain Question Answeringcited by 11
  20. 2021SDR: Efficient Neural Re-ranking using Succinct Document Representationcited by 7

Built on by

Not cited yet.