mlsanityData to 14 Oct 2026

UnifieR: A Unified Retriever for Large-Scale Retrieval

arXiv 2205.11194 ↗First posted 23 May 2022cs.IR cs.CLcited by 19

Not on the map. Why?

Abstract (arXiv)

Large-scale retrieval is to recall relevant documents from a huge collection given a query. It relies on representation learning to embed documents and queries into a common semantic encoding space. According to the encoding space, recent retrieval methods based on pre-trained language models (PLM) can be coarsely categorized into either dense-vector or lexicon-based paradigms. These two paradigms unveil the PLMs' representation capability in different granularities, i.e., global sequence-level compression and local word-level contexts, respectively. Inspired by their complementary global-local contextualization and distinct representing views, we propose a new learning framework, UnifieR which unifies dense-vector and lexicon-based retrieval in one model with a dual-representing capability. Experiments on passage retrieval benchmarks verify its effectiveness in both paradigms. A uni-retrieval scheme is further presented with even better retrieval quality. We lastly evaluate the model on BEIR benchmark to verify its transferability.

Key facts. Cited by 19 papers in the map (data through 14 Oct 2026). Builds on 60 of its 83 references that are in the map. First posted on arXiv 23 May 2022.

Builds on

60 of 83 references.

References · 60

  1. 2017Attention Is All You Needcited by 60,361
  2. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,399
  3. 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,599
  4. 2019RoBERTa: A Robustly Optimized BERT Pretraining Approachcited by 13,372
  5. 2019Sentence-BERT: Sentence Embeddings using Siamese BERT-Networkscited by 9,170
  6. 2020Dense Passage Retrieval for Open-Domain Question Answeringcited by 3,824
  7. 2020Longformer: The Long-Document Transformercited by 3,413
  8. 2014Sequence to Sequence Learning with Neural Networkscited by 2,986
  9. 2019ALBERT: A Lite BERT for Self-supervised Learning of Language Representationscited by 2,790
  10. 2021SimCSE: Simple Contrastive Learning of Sentence Embeddingscited by 2,173
  11. 2016MS MARCO: A Human Generated MAchine Reading COmprehension Datasetcited by 1,605
  12. 2021BEIR: A Heterogenous Benchmark for Zero-shot Evaluation of Information Retrieval Modelscited by 1,252
38 more
  1. 2020ColBERT: Efficient and Effective Passage Search via Contextualized Late Interaction over BERTcited by 1,178
  2. 2018Co-teaching: Robust Training of Deep Neural Networks with Extremely Noisy Labelscited by 1,059
  3. 2019RotatE: Knowledge Graph Embedding by Relational Rotation in Complex Spacecited by 898
  4. 2020Approximate Nearest Neighbor Negative Contrastive Learning for Dense Text Retrievalcited by 849
  5. 2017Reading Wikipedia to Answer Open-Domain Questionscited by 777
  6. 2019Unified Language Model Pre-training for Natural Language Understanding and Generationcited by 670
  7. 2020UnifiedQA: Crossing Format Boundaries With a Single QA Systemcited by 476
  8. 2021ColBERTv2: Effective and Efficient Retrieval via Lightweight Late Interactioncited by 476
  9. 2021Large Dual Encoders Are Generalizable Retrieverscited by 456
  10. 2020RocketQA: An Optimized Training Approach to Dense Passage Retrieval for Open-Domain Question Answeringcited by 413
  11. 2019Multi-Task Deep Neural Networks for Natural Language Understandingcited by 350
  12. 2020Overview of the TREC 2019 deep learning trackcited by 286
  13. 2021SPLADE: Sparse Lexical and Expansion Model for First Stage Rankingcited by 286
  14. 2020UNIMO: Towards Unified-Modal Understanding and Generation via Cross-Modal Contrastive Learningcited by 239
  15. 2021Efficiently Teaching an Effective Dense Retriever with Balanced Topic Aware Samplingcited by 238
  16. 2019Document Expansion by Query Predictioncited by 231
  17. 2022UnifiedSKG: Unifying and Multi-Tasking Structured Knowledge Grounding with Text-to-Text Language Modelscited by 231
  18. 2022SGPT: GPT Sentence Embeddings for Semantic Searchcited by 204
  19. 2021Unsupervised Corpus Aware Language Model Pre-training for Dense Passage Retrievalcited by 203
  20. 2021R-Drop: Regularized Dropout for Neural Networkscited by 202
  21. 2021RocketQAv2: A Joint Training Method for Dense Passage Retrieval and Passage Re-rankingcited by 176
  22. 2019Poly-encoders: Transformer Architectures and Pre-training Strategies for Fast and Accurate Multi-sentence Scoringcited by 165
  23. 2021Condenser: a Pre-training Architecture for Dense Retrievalcited by 162
  24. 2021Optimizing Dense Retrieval Model Training with Hard Negativescited by 155
  25. 2020Improving Efficient Neural Ranking Models with Cross-Architecture Knowledge Distillationcited by 145
  26. 2022DiffCSE: Difference-based Contrastive Learning for Sentence Embeddingscited by 132
  27. 2021SPLADE v2: Sparse Lexical and Expansion Model for Information Retrievalcited by 127
  28. 2020A Survey of Knowledge-Enhanced Text Generationcited by 120
  29. 2021COIL: Revisit Exact Lexical Match in Information Retrieval with Contextualized Inverted Listcited by 113
  30. 2022From Distillation to Hard Negative Sampling: Making Sparse Neural IR Models More Effectivecited by 98
  31. 2020Fast WordPiece Tokenizationcited by 87
  32. 2021Adversarial Retriever-Ranker for dense text retrievalcited by 83
  33. 2021A Few Brief Notes on DeepImpact, COIL, and a Conceptual Framework for Information Retrieval Techniquescited by 69
  34. 2019Context-Aware Sentence/Passage Term Importance Estimation For First Stage Retrievalcited by 65
  35. 2021Salient Phrase Aware Dense Retrieval: Can a Dense Retriever Imitate a Sparse One?cited by 60
  36. 2021Semantic Models for the First-stage Retrieval: A Comprehensive Reviewcited by 57
  37. 2022ZeroPrompt: Scaling Prompt-Based Pretraining to 1,000 Tasks Improves Zero-Shot Generalizationcited by 54
  38. 2021PAIR: Leveraging Passage-Centric Similarity Relation for Improving Dense Passage Retrievalcited by 50

50 most cited of 60 All 60 (JSON)

Built on by

Cited by year

Citing papers per year: 2022: 5; 2023: 4; 2024: 5; 2025: 5.

19 papers.

Sort:

By area:

  1. 2024LLMs are Also Effective Embedding Models: An In-depth Overviewcited by 32
  2. 2022CITADEL: Conditional Token Interaction via Dynamic Lexical Routing for Efficient and Effective Multi-Vector Retrievalcited by 19
  3. 2024UMIE: Unified Multimodal Information Extraction with Instruction Tuningcited by 17
  4. 2022LED: Lexicon-Enlightened Dense Retriever for Large-Scale Retrievalcited by 15
  5. 2025Uni-Retrieval: A Multi-Style Retrieval Framework for STEM's Educationcited by 13
  6. 2025Mining Intrinsic Rewards from LLM Hidden States for Efficient Best-of-N Samplingcited by 9
  7. 2023CAME: Competitively Learning a Mixture-of-Experts Model for First-stage Retrievalcited by 6
  8. 2025Enhancing Lexicon-Based Text Embeddings with Large Language Modelscited by 6
  9. 2022Retrieval-based Disentangled Representation Learning with Natural Language Supervisioncited by 5
  10. 2023Noisy Pair Corrector for Dense Retrievalcited by 5
  11. 2022Adam: Dense Retrieval Distillation with Adaptive Dark Examplescited by 4
  12. 2024Multi-Field Adaptive Retrievalcited by 4
7 more
  1. 2023Challenging Decoder helps in Masked Auto-Encoder Pre-training for Dense Passage Retrievalcited by 3
  2. 2022Hybrid Inverted Index Is a Robust Accelerator for Dense Retrievalcited by 2
  3. 2025From Query to Explanation: Uni-RAG for Multi-Modal Retrieval-Augmented Learning in STEMcited by 2
  4. 2023LexLIP: Lexicon-Bottlenecked Language-Image Pre-Training for Large-Scale Image-Text Retrievalcited by 1
  5. 2024Unifying Latent and Lexicon Representations for Effective Video-Text Retrievalcited by 1
  6. 2024Mitigating Hallucinations Using Ensemble of Knowledge Graph and Vector Store in Large Language Models to Enhance Mental Health Supportcited by 1
  7. 2025HyReC: Exploring Hybrid-based Retriever for Chinesecited by 0