mlsanityData to 7 Oct 2026

Learning Diverse Document Representations with Deep Query Interactions for Dense Retrieval

arXiv 2208.04232 ↗First posted 8 Aug 2022cs.IR cs.CLcited by 4

Not on the map. Why?

Abstract (arXiv)

In this paper, we propose a new dense retrieval model which learns diverse document representations with deep query interactions. Our model encodes each document with a set of generated pseudo-queries to get query-informed, multi-view document representations. It not only enjoys high inference efficiency like the vanilla dual-encoder models, but also enables deep query-document interactions in document encoding and provides multi-faceted representations to better match different queries. Experiments on several benchmarks demonstrate the effectiveness of the proposed method, out-performing strong dual encoder baselines.The code is available at urlhttps://github.com/jordane95/dual-cross-encoder

Key facts. Cited by 4 papers in the map (data through 7 Oct 2026). Builds on 29 of its 33 references that are in the map. First posted on arXiv 8 Aug 2022.

Builds on

29 of 33 references.

References · 29

  1. 2017Attention Is All You Needcited by 59,932
  2. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  3. 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,491
  4. 2015Distilling the Knowledge in a Neural Networkcited by 10,076
  5. 2020Dense Passage Retrieval for Open-Domain Question Answeringcited by 3,779
  6. 2017Billion-scale similarity search with GPUscited by 2,477
  7. 2016MS MARCO: A Human Generated MAchine Reading COmprehension Datasetcited by 1,591
  8. 2018Hierarchical Neural Story Generationcited by 1,182
  9. 2020ColBERT: Efficient and Effective Passage Search via Contextualized Late Interaction over BERTcited by 1,160
  10. 2020Approximate Nearest Neighbor Negative Contrastive Learning for Dense Text Retrievalcited by 846
  11. 2017Reading Wikipedia to Answer Open-Domain Questionscited by 777
  12. 2019Passage Re-ranking with BERTcited by 629
17 more
  1. 2018Texygen: A Benchmarking Platform for Text Generation Modelscited by 484
  2. 2020RocketQA: An Optimized Training Approach to Dense Passage Retrieval for Open-Domain Question Answeringcited by 410
  3. 2020Overview of the TREC 2019 deep learning trackcited by 282
  4. 2021Efficiently Teaching an Effective Dense Retriever with Balanced Topic Aware Samplingcited by 238
  5. 2019Document Expansion by Query Predictioncited by 225
  6. 2020Sparse, Dense, and Attentional Representations for Text Retrievalcited by 213
  7. 2021Overview of the TREC 2020 deep learning trackcited by 193
  8. 2021RocketQAv2: A Joint Training Method for Dense Passage Retrieval and Passage Re-rankingcited by 175
  9. 2019Poly-encoders: Transformer Architectures and Pre-training Strategies for Fast and Accurate Multi-sentence Scoringcited by 164
  10. 2021Optimizing Dense Retrieval Model Training with Hard Negativescited by 154
  11. 2021GPL: Generative Pseudo Labeling for Unsupervised Domain Adaptation of Dense Retrievalcited by 119
  12. 2021Learning Passage Impacts for Inverted Indexescited by 68
  13. 2022Tevatron: An Efficient and Flexible Toolkit for Dense Retrievalcited by 47
  14. 2020Efficient Document Re-Ranking for Transformers by Precomputing Term Representationscited by 44
  15. 2022Multi-View Document Representation Learning for Open-Domain Dense Retrievalcited by 41
  16. 2022ERNIE-Search: Bridging Cross-Encoder with Dual-Encoder via Self On-the-fly Distillation for Dense Passage Retrievalcited by 41
  17. 2021Improving Document Representations by Generating Pseudo Query Embeddings for Dense Retrievalcited by 14

Built on by

Cited by year

Citing papers per year: 2022: 2; 2023: 0; 2024: 2.

4 papers.

  1. 2024Generative Dense Retrieval: Memory Can Be a Burdencited by 11
  2. 2022Query-as-context Pre-training for Dense Passage Retrievalcited by 9
  3. 2022CAPSTONE: Curriculum Sampling for Dense Retrieval with Document Expansioncited by 4
  4. 2024QAEncoder: Towards Aligned Representation Learning in Question Answering Systemscited by 3