mlsanityData to 14 Oct 2026

InPars-Light: Cost-Effective Unsupervised Training of Efficient Rankers

arXiv 2301.02998 ↗First posted 8 Jan 2023cs.IR cs.AI cs.CLcited by 14

Not on the map. Why?

Abstract (arXiv)

We carried out a reproducibility study of InPars, which is a method for unsupervised training of neural rankers (Bonifacio et al., 2022). As a by-product, we developed InPars-light, which is a simple-yet-effective modification of InPars. Unlike InPars, InPars-light uses 7x-100x smaller ranking models and only a freely available language model BLOOM, which -- as we found out -- produced more accurate rankers compared to a proprietary GPT-3 model. On all five English retrieval collections (used in the original InPars study) we obtained substantial (7%-30%) and statistically significant improvements over BM25 (in nDCG and MRR) using only a 30M parameter six-layer MiniLM-30M ranker and a single three-shot prompt. In contrast, in the InPars study only a 100x larger monoT5-3B model consistently outperformed BM25, whereas their smaller monoT5-220M model (which is still 7x larger than our MiniLM ranker) outperformed BM25 only on MS MARCO and TREC DL 2020. In the same three-shot prompting scenario, our 435M parameter DeBERTA v3 ranker was at par with the 7x larger monoT5-3B (average gain over BM25 of 1.3 vs 1.32): In fact, on three out of five datasets, DeBERTA slightly outperformed monoT5-3B. Finally, these good results were achieved by re-ranking only 100 candidate documents compared to 1000 used by Bonifacio et al. (2022). We believe that InPars-light is the first truly cost-effective prompt-based unsupervised recipe to train and deploy neural ranking models that outperform BM25. Our code and data is publicly available. https://github.com/searchivarius/inpars_light/

Key facts. Cited by 14 papers in the map (data through 14 Oct 2026). Builds on 37 of its 57 references that are in the map. First posted on arXiv 8 Jan 2023.

Builds on

37 of 57 references.

References · 37

  1. 2017Attention Is All You Needcited by 60,361
  2. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,399
  3. 2020Language Models are Few-Shot Learnerscited by 32,419
  4. 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,599
  5. 2019RoBERTa: A Robustly Optimized BERT Pretraining Approachcited by 13,372
  6. 2015Distilling the Knowledge in a Neural Networkcited by 10,324
  7. 2021Pre-train, Prompt, and Predict: A Systematic Survey of Prompting Methods in Natural Language Processingcited by 2,234
  8. 2020DeBERTa: Decoding-enhanced BERT with Disentangled Attentioncited by 2,181
  9. 2022BLOOM: A 176B-Parameter Open-Access Multilingual Language Modelcited by 1,984
  10. 2014FitNets: Hints for Thin Deep Netscited by 1,653
  11. 2016MS MARCO: A Human Generated MAchine Reading COmprehension Datasetcited by 1,605
  12. 2020MiniLM: Deep Self-Attention Distillation for Task-Agnostic Compression of Pre-Trained Transformerscited by 1,463
25 more
  1. 2021BEIR: A Heterogenous Benchmark for Zero-shot Evaluation of Information Retrieval Modelscited by 1,252
  2. 2019Passage Re-ranking with BERTcited by 645
  3. 2015Cyclical Learning Rates for Training Neural Networkscited by 629
  4. 2022Precise Zero-Shot Dense Retrieval without Relevance Labelscited by 517
  5. 2021Large Dual Encoders Are Generalizable Retrieverscited by 456
  6. 2020Document Ranking with a Pretrained Sequence-to-Sequence Modelcited by 376
  7. 2020Keystroke Dynamics as Part of Lifeloggingcited by 334
  8. 2020Overview of the TREC 2019 deep learning trackcited by 286
  9. 2020Pretrained Transformers for Text Ranking: BERT and Beyondcited by 267
  10. 2020On the Stability of Fine-tuning BERT: Misconceptions, Explanations, and Strong Baselinescited by 255
  11. 2022Promptagator: Few-shot Dense Retrieval From 8 Examplescited by 235
  12. 2019ERNIE 2.0: A Continual Pre-training Framework for Language Understandingcited by 231
  13. 2021Generating Datasets with Pretrained Language Modelscited by 193
  14. 2022Improving Passage Retrieval with Zero-Shot Question Generationcited by 165
  15. 2020Improving Efficient Neural Ranking Models with Cross-Architecture Knowledge Distillationcited by 145
  16. 2019Synthetic QA Corpora Generation with Roundtrip Consistencycited by 143
  17. 2021GPL: Generative Pseudo Labeling for Unsupervised Domain Adaptation of Dense Retrievalcited by 119
  18. 2023InPars-v2: Large Language Models as Efficient Dataset Generators for Information Retrievalcited by 73
  19. 2020Distilling Dense Representations for Ranking using Tightly-Coupled Teacherscited by 52
  20. 2021Simplified Data Wrangling with ir_datasetscited by 27
  21. 2022In Defense of Cross-Encoders for Zero-Shot Retrievalcited by 21
  22. 2020Flexible retrieval with NMSLIB and FlexNeuARTcited by 14
  23. 2021A Systematic Evaluation of Transfer Learning and Pseudo-labeling with BERT-based Ranking Modelscited by 8
  24. 2022Synthetic Target Domain Supervision for Open Retrieval QAcited by 6
  25. 2022Domain Adaptation for Dense Retrieval through Self-Supervision by Pseudo-Relevance Labelingcited by 2

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2023: 8; 2024: 2; 2025: 3; 2026: 1.

14 papers.

Sort:

By area:

  1. 2023Large Language Models for Information Retrieval: A Surveycited by 331
  2. 2023Zero-Shot Listwise Document Reranking with a Large Language Modelcited by 82
  3. 2023RankVicuna: Zero-Shot Listwise Document Reranking with Open-Source Large Language Modelscited by 70
  4. 2023Large Language Model Augmented Narrative Driven Recommendationscited by 31
  5. 2023Large Language Models are Strong Zero-Shot Retrievercited by 26
  6. 2023Leveraging LLMs for Synthesizing Training Data Across Many Languages in Multilingual Dense Retrievalcited by 24
  7. 2023Soft Prompt Tuning for Augmenting Dense Retrieval with Large Language Modelscited by 12
  8. 2024GenSco: Can Question Decomposition based Passage Alignment improve Question Answering?cited by 3
  9. 2024Disentangling Questions from Query Generation for Task-Adaptive Retrievalcited by 3
  10. 2023InPars Toolkit: A Unified and Reproducible Synthetic Data Generation Pipeline for Neural Information Retrievalcited by 2
  11. 2025Study on LLMs for Promptagator-Style Dense Retriever Trainingcited by 2
  12. 2026LLM-Confidence Reranker: A Training-Free Approach for Enhancing Retrieval-Augmented Generation Systemscited by 2
2 more
  1. 2025LLM-guided Plan and Retrieval: A Strategic Alignment for Interpretable User Satisfaction Estimation in Dialoguecited by 1
  2. 2025LMAR: Language Model Augmented Retriever for Domain-specific Knowledge Indexingcited by 1