mlsanityData to 14 Oct 2026

The Expando-Mono-Duo Design Pattern for Text Ranking with Pretrained Sequence-to-Sequence Models

arXiv 2101.05667 ↗First posted 14 Jan 2021cs.IR cs.CLcited by 81

Not on the map. Why?

Abstract (arXiv)

We propose a design pattern for tackling text ranking problems, dubbed "Expando-Mono-Duo", that has been empirically validated for a number of ad hoc retrieval tasks in different domains. At the core, our design relies on pretrained sequence-to-sequence models within a standard multi-stage ranking architecture. "Expando" refers to the use of document expansion techniques to enrich keyword representations of texts prior to inverted indexing. "Mono" and "Duo" refer to components in a reranking pipeline based on a pointwise model and a pairwise model that rerank initial candidates retrieved using keyword search. We present experimental results from the MS MARCO passage and document ranking tasks, the TREC 2020 Deep Learning Track, and the TREC-COVID challenge that validate our design. In all these tasks, we achieve effectiveness that is at or near the state of the art, in some cases using a zero-shot approach that does not exploit any training data from the target task. To support replicability, implementations of our design pattern are open-sourced in the Pyserini IR toolkit and PyGaggle neural reranking library.

Key facts. Cited by 81 papers in the map (data through 14 Oct 2026). Builds on 35 of its 61 references that are in the map. First posted on arXiv 14 Jan 2021.

Builds on

35 of 61 references.

References · 35

  1. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,399
  2. 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,599
  3. 2019BART: Denoising Sequence-to-Sequence Pre-training for Natural Language Generation, Translation, and Comprehensioncited by 5,893
  4. 2020Dense Passage Retrieval for Open-Domain Question Answeringcited by 3,824
  5. 2020Reformer: The Efficient Transformercited by 1,668
  6. 2016MS MARCO: A Human Generated MAchine Reading COmprehension Datasetcited by 1,605
  7. 2018Hierarchical Neural Story Generationcited by 1,193
  8. 2020ColBERT: Efficient and Effective Passage Search via Contextualized Late Interaction over BERTcited by 1,178
  9. 2018Self-Attention with Relative Position Representationscited by 1,120
  10. 2019PEGASUS: Pre-training with Extracted Gap-sentences for Abstractive Summarizationcited by 1,102
  11. 2020Approximate Nearest Neighbor Negative Contrastive Learning for Dense Text Retrievalcited by 849
  12. 2019Latent Retrieval for Weakly Supervised Open Domain Question Answeringcited by 695
23 more
  1. 2019Passage Re-ranking with BERTcited by 645
  2. 2020RocketQA: An Optimized Training Approach to Dense Passage Retrieval for Open-Domain Question Answeringcited by 413
  3. 2020Document Ranking with a Pretrained Sequence-to-Sequence Modelcited by 376
  4. 2020Pretrained Transformers for Text Ranking: BERT and Beyondcited by 267
  5. 2019ScispaCy: Fast and Robust Models for Biomedical Natural Language Processingcited by 244
  6. 2019Document Expansion by Query Predictioncited by 231
  7. 2019Multi-Stage Document Ranking with BERTcited by 201
  8. 2020CORD-19: The COVID-19 Open Research Datasetcited by 180
  9. 2017A Deep Relevance Matching Model for Ad-hoc Retrievalcited by 126
  10. 2019Deeper Text Understanding for IR with Contextual Neural Language Modelingcited by 113
  11. 2017End-to-End Neural Ad-hoc Ranking with Kernel Poolingcited by 85
  12. 2019CEDR: Contextualized Embeddings for Document Rankingcited by 77
  13. 2016Learning to Match Using Local and Distributed Representations of Text for Web Searchcited by 68
  14. 2019Context-Aware Sentence/Passage Term Importance Estimation For First Stage Retrievalcited by 65
  15. 2020Distilling Dense Representations for Ranking using Tightly-Coupled Teacherscited by 52
  16. 2020Learning-to-Rank with BERT in TF-Rankingcited by 36
  17. 2017Cascade Ranking for Operational E-commerce Searchcited by 29
  18. 2020Complementing Lexical Retrieval with Semantic Residual Embeddingcited by 27
  19. 2019Critically Examining the "Neural Hype": Weak Baselines and the Additivity of Effectiveness Gains from Neural Ranking Modelscited by 24
  20. 2020The Cascade Transformer: an Application for Efficient Answer Sentence Selectioncited by 20
  21. 2020Covidex: Neural Ranking Models and Keyword Search Infrastructure for the COVID-19 Open Research Datasetcited by 15
  22. 2020SLEDGE-Z: A Zero-Shot Baseline for COVID-19 Literature Searchcited by 6
  23. 2017Efficient and Effective Tail Latency Minimization in Multi-Stage Retrieval Systemscited by 5

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2021: 4; 2022: 10; 2023: 10; 2024: 27; 2025: 18; 2026: 12.

81 papers.

Sort:

By area:

  1. 2023Large Language Models for Information Retrieval: A Surveycited by 331
  2. 2021Efficiently Teaching an Effective Dense Retriever with Balanced Topic Aware Samplingcited by 238
  3. 2021mMARCO: A Multilingual Version of the MS MARCO Passage Ranking Datasetcited by 116
  4. 2022RankT5: Fine-Tuning T5 for Text Ranking with Ranking Lossescited by 115
  5. 2023RankVicuna: Zero-Shot Listwise Document Reranking with Open-Source Large Language Modelscited by 70
  6. 2022FiD-Light: Efficient and Effective Retrieval-Augmented Text Generationcited by 63
  7. 2023Leveraging Large Language Models in Conversational Recommender Systemscited by 63
  8. 2022InPars: Data Augmentation for Information Retrieval using Large Language Modelscited by 62
  9. 2023A Setwise Approach for Effective and Highly Efficient Zero-shot Ranking with Large Language Modelscited by 55
  10. 2023How Does Generative Retrieval Scale to Millions of Passages?cited by 48
  11. 2024TourRank: Utilizing Large Language Models for Documents Ranking with a Tournament-Inspired Strategycited by 47
  12. 2022Order-Disorder: Imitation Adversarial Attacks for Black-box Neural Ranking Modelscited by 37
38 more
  1. 2025A Survey of Multimodal Retrieval-Augmented Generationcited by 37
  2. 2024JudgeRank: Leveraging Large Language Models for Reasoning-Intensive Rerankingcited by 34
  3. 2024Ragnar"ok: A Reusable RAG Framework and Baselines for TREC 2024 Retrieval-Augmented Generation Trackcited by 32
  4. 2025Rank-K: Test-Time Reasoning for Listwise Rerankingcited by 29
  5. 2024Initial Nugget Evaluation Results for the TREC 2024 RAG Track with the AutoNuggetizer Frameworkcited by 28
  6. 2024Understand What LLM Needs: Dual Preference Alignment for Retrieval-Augmented Generationcited by 27
  7. 2024GenQREnsemble: Zero-Shot LLM Ensemble Prompting for Generative Query Reformulationcited by 23
  8. 2022Open Domain Multi-document Summarization: A Comprehensive Study of Model Brittleness under Retrievalcited by 20
  9. 2024Leveraging Passage Embeddings for Efficient Listwise Reranking with Large Language Modelscited by 20
  10. 2024Planning Ahead in Generative Retrieval: Guiding Autoregressive Generation through Simultaneous Decodingcited by 19
  11. 2025Towards AI Search Paradigmcited by 19
  12. 2024Query Performance Prediction using Relevance Judgments Generated by Large Language Modelscited by 18
  13. 2024Do Large Language Models Rank Fairly? An Empirical Study on the Fairness of LLMs as Rankerscited by 17
  14. 2024LLM-RankFusion: Mitigating Intrinsic Inconsistency in LLM-based Rankingcited by 16
  15. 2023Resources for Brewing BEIR: Reproducible Reference Models and an Official Leaderboardcited by 15
  16. 2025Rankers, Judges, and Assistants: Towards Understanding the Interplay of LLMs in Information Retrieval Evaluationcited by 15
  17. 2023Rank-without-GPT: Building GPT-Independent Listwise Rerankers on Open-Source Large Language Modelscited by 14
  18. 2021Answer Generation for Retrieval-based Question Answering Systemscited by 13
  19. 2024Enhancing Post-Hoc Attributions in Long Document Comprehension via Coarse Grained Answer Decompositioncited by 12
  20. 2025LLM4Ranking: An Easy-to-use Framework of Utilizing Large Language Models for Document Rerankingcited by 12
  21. 2025DeAR: Dual-Stage Document Reranking with Reasoning Agents via LLM Distillationcited by 11
  22. 2021Generating Usage-related Questions for Preference Elicitation in Conversational Recommender Systemscited by 10
  23. 2024Unbiased Learning to Rank Meets Reality: Lessons from Baidu's Large-Scale Search Datasetcited by 10
  24. 2024State Space Models are Strong Text Rerankerscited by 10
  25. 2024ReasoningRank: Teaching Student Models to Rank through Reasoning-Based Knowledge Distillationcited by 9
  26. 2025DocReRank: Single-Page Hard Negative Query Generation for Training Multi-Modal RAG Rerankerscited by 9
  27. 2025Are LLMs Reliable Rankers? Rank Manipulation via Two-Stage Token Optimizationcited by 9
  28. 2025AcuRank: Uncertainty-Aware Adaptive Computation for Listwise Rerankingcited by 8
  29. 2022HYRR: Hybrid Infused Reranking for Passage Retrievalcited by 6
  30. 2023Explain like I am BM25: Interpreting a Dense Model's Ranked-List with a Sparse Approximationcited by 5
  31. 2024Generative Query Reformulation Using Ensemble Prompting, Document Fusion, and Relevance Feedbackcited by 5
  32. 2025Constructing and Evaluating Declarative RAG Pipelines in PyTerriercited by 5
  33. 2026Optimizing User Profiles via Contextual Bandits for Retrieval-Augmented LLM Personalizationcited by 5
  34. 2022Learning List-Level Domain-Invariant Representations for Rankingcited by 4
  35. 2024Estimating the Usefulness of Clarifying Questions and Answers for Conversational Searchcited by 4
  36. 2024Q-PEFT: Query-dependent Parameter Efficient Fine-tuning for Text Reranking with Large Language Modelscited by 4
  37. 2024Few-shot Prompting for Pairwise Ranking: An Effective Non-Parametric Retrieval Modelcited by 4
  38. 2022Continuous Active Learning Using Pretrained Transformerscited by 3

50 most cited of 81 All 81 (JSON)