mlsanityData to 7 Oct 2026

InPars: Data Augmentation for Information Retrieval using Large Language Models

arXiv 2202.05144 ↗First posted 10 Feb 2022cs.CLcited by 60

Not on the map. Why?

Abstract (arXiv)

The information retrieval community has recently witnessed a revolution due to large pretrained transformer models. Another key ingredient for this revolution was the MS MARCO dataset, whose scale and diversity has enabled zero-shot transfer learning to various tasks. However, not all IR tasks and domains can benefit from one single dataset equally. Extensive research in various NLP tasks has shown that using domain-specific training data, as opposed to a general-purpose one, improves the performance of neural models. In this work, we harness the few-shot capabilities of large pretrained language models as synthetic data generators for IR tasks. We show that models finetuned solely on our unsupervised dataset outperform strong baselines such as BM25 as well as recently proposed self-supervised dense retrieval methods. Furthermore, retrievers finetuned on both supervised and our synthetic data achieve better zero-shot transfer than models finetuned only on supervised data. Code, models, and data are available at https://github.com/zetaalphavector/inpars .

Key facts. Cited by 60 papers in the map (data through 7 Oct 2026). Builds on 39 of its 54 references that are in the map. First posted on arXiv 10 Feb 2022.

Builds on

39 of 54 references.

References · 39

  1. 2020Language Models are Few-Shot Learnerscited by 32,199
  2. 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,491
  3. 2020Dense Passage Retrieval for Open-Domain Question Answeringcited by 3,779
  4. 2021Finetuned Language Models Are Zero-Shot Learnerscited by 3,747
  5. 2019The Curious Case of Neural Text Degenerationcited by 2,881
  6. 2017Billion-scale similarity search with GPUscited by 2,477
  7. 2016MS MARCO: A Human Generated MAchine Reading COmprehension Datasetcited by 1,591
  8. 2021Multitask Prompted Training Enables Zero-Shot Task Generalizationcited by 1,462
  9. 2021BEIR: A Heterogenous Benchmark for Zero-shot Evaluation of Information Retrieval Modelscited by 1,225
  10. 2021Scaling Language Models: Methods, Analysis & Insights from Training Gophercited by 1,199
  11. 2020ColBERT: Efficient and Effective Passage Search via Contextualized Late Interaction over BERTcited by 1,160
  12. 2020It's Not Just Size That Matters: Small Language Models Are Also Few-Shot Learnerscited by 668
27 more
  1. 2021ColBERTv2: Effective and Efficient Retrieval via Lightweight Late Interactioncited by 464
  2. 2020Document Ranking with a Pretrained Sequence-to-Sequence Modelcited by 373
  3. 2022Text and Code Embeddings by Contrastive Pre-Trainingcited by 358
  4. 2021Unsupervised Corpus Aware Language Model Pre-training for Dense Passage Retrievalcited by 203
  5. 2018Contextual Augmentation: Data Augmentation by Words with Paradigmatic Relationscited by 201
  6. 2022WANLI: Worker and AI Collaboration for Natural Language Inference Dataset Creationcited by 200
  7. 2022Generating Training Data with Language Models: Towards Zero-Shot Language Understandingcited by 199
  8. 2021Overview of the TREC 2020 deep learning trackcited by 193
  9. 2021Generating Datasets with Pretrained Language Modelscited by 192
  10. 2019Not Enough Data? Deep Learning to the Rescue!cited by 174
  11. 2020Data Augmentation using Pre-trained Transformer Modelscited by 168
  12. 2017Data Augmentation for Low-Resource Neural Machine Translationcited by 139
  13. 2021GPL: Generative Pseudo Labeling for Unsupervised Domain Adaptation of Dense Retrievalcited by 119
  14. 2019Deeper Text Understanding for IR with Contextual Neural Language Modelingcited by 111
  15. 2021Language Models are Few-shot Multilingual Learnerscited by 92
  16. 2021The Expando-Mono-Duo Design Pattern for Text Ranking with Pretrained Sequence-to-Sequence Modelscited by 81
  17. 2021AdaptSum: Towards Low-Resource Domain Adaptation for Abstractive Summarizationcited by 51
  18. 2020Generative Data Augmentation for Commonsense Reasoningcited by 49
  19. 2020The Curse of Dense Low-Dimensional Information Retrieval for Large Index Sizescited by 48
  20. 2021Learning to Retrieve Passages without Supervisioncited by 46
  21. 2020PARADE: Passage Representation Aggregation for Document Rerankingcited by 44
  22. 2020DARE: Data Augmented Relation Extraction with GPT-2cited by 43
  23. 2017Cascade Ranking for Operational E-commerce Searchcited by 29
  24. 2020Simulated Chats for Building Dialog Systems: Learning to Generate Conversations from Instructionscited by 25
  25. 2021Unsupervised Neural Machine Translation with Generative Language Models Onlycited by 24
  26. 2020On the Reliability of Test Collections for Evaluating Systems of Different Typescited by 4
  27. 2021Selecting Parallel In-domain Sentences for Neural Machine Translation Using Monolingual Textscited by 4

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2022: 5; 2023: 21; 2024: 17; 2025: 7; 2026: 10.

60 papers.

Sort:

By area:

  1. 2023Is ChatGPT Good at Search? Investigating Large Language Models as Re-Ranking Agentscited by 356
  2. 2023Large Language Models for Information Retrieval: A Surveycited by 331
  3. 2024A Survey on Knowledge Distillation of Large Language Modelscited by 276
  4. 2022SGPT: GPT Sentence Embeddings for Semantic Searchcited by 204
  5. 2024Gecko: Versatile Text Embeddings Distilled from Large Language Modelscited by 81
  6. 2023InPars-v2: Large Language Models as Efficient Dataset Generators for Information Retrievalcited by 73
  7. 2023AI-Generated Content (AIGC): A Surveycited by 59
  8. 2022Language Models in the Loop: Incorporating Prompting into Weak Supervisioncited by 51
  9. 2022Questions Are All You Need to Train a Dense Passage Retrievercited by 47
  10. 2024Promptriever: Instruction-Trained Retrievers Can Be Prompted Like Language Modelscited by 39
  11. 2023UDAPDR: Unsupervised Domain Adaptation via LLM Prompting and Distillation of Rerankerscited by 37
  12. 2022DSI++: Updating Transformer Memory with New Documentscited by 35
38 more
  1. 2024A Survey on Data Augmentation in Large Model Eracited by 31
  2. 2024Understand What LLM Needs: Dual Preference Alignment for Retrieval-Augmented Generationcited by 27
  3. 2025RaDeR: Reasoning-aware Dense Retrieval Modelscited by 27
  4. 2023Iterative Zero-Shot LLM Prompting for Knowledge Graph Constructioncited by 25
  5. 2024I Think, Therefore I am: Benchmarking Awareness of Large Language Models Using AwareBenchcited by 20
  6. 2025DRAMA: Diverse Augmentation from Large Language Models to Smaller Dense Retrieverscited by 19
  7. 2025Rankers, Judges, and Assistants: Towards Understanding the Interplay of LLMs in Information Retrieval Evaluationcited by 15
  8. 2023UMASS_BioNLP at MEDIQA-Chat 2023: Can LLMs generate high-quality synthetic note-oriented doctor-patient conversations?cited by 14
  9. 2023Large Language Models as Batteries-Included Zero-Shot ESCO Skills Matcherscited by 14
  10. 2023Context Aware Query Rewriting for Text Rankers using LLMcited by 14
  11. 2024Automated data processing and feature engineering for deep learning and big data applications: a surveycited by 14
  12. 2023IncDSI: Incrementally Updatable Document Retrievalcited by 12
  13. 2025GLaD: Geometric Latent Distillation for Vision-Language-Action Modelscited by 10
  14. 2023Exploring the Viability of Synthetic Query Generation for Relevance Predictioncited by 9
  15. 2023Dense Retrieval Adaptation using Target Domain Descriptioncited by 8
  16. 2023cTBLS: Augmenting Large Language Models with Conversational Tablescited by 7
  17. 2023Selecting which Dense Retriever to use for Zero-Shot Searchcited by 7
  18. 2024Enhancing Question Answering for Enterprise Knowledge Bases using Large Language Modelscited by 7
  19. 2022HYRR: Hybrid Infused Reranking for Passage Retrievalcited by 6
  20. 2024Generalized Contrastive Learning for Multi-Modal Retrieval and Rankingcited by 6
  21. 2024Meta Knowledge for Retrieval Augmented Large Language Modelscited by 6
  22. 2024A Large Language Model and Denoising Diffusion Framework for Targeted Design of Microstructures with Commands in Natural Languagecited by 6
  23. 2023Synthetic Imitation Edit Feedback for Factual Alignment in Clinical Summarizationcited by 5
  24. 2024Self-Retrieval: End-to-End Information Retrieval with One Large Language Modelcited by 4
  25. 2024Is Your AI Truly Yours? Leveraging Blockchain for Copyrights, Provenance, and Lineagecited by 4
  26. 2025CTRL-Rec: Controlling Recommender Systems With Natural Languagecited by 4
  27. 2026Are LLM-Based Retrievers Worth Their Cost? An Empirical Study of Efficiency, Robustness, and Reasoning Overheadcited by 4
  28. 2024Overcoming LLM Challenges using RAG-Driven Precision in Coffee Leaf Disease Remediationcited by 3
  29. 2025Don't Retrieve, Generate: Prompting LLMs for Synthetic Training Data in Dense Retrievalcited by 3
  30. 2023Larger Probes Tell a Different Story: Extending Psycholinguistic Datasets Via In-Context Learningcited by 2
  31. 2023Overview of the TREC 2023 Product Product Search Trackcited by 2
  32. 2024A Self-feedback Knowledge Elicitation Approach for Chemical Reaction Predictionscited by 2
  33. 2023Relation Extraction in underexplored biomedical domains: A diversity-optimised sampling and synthetic data generation approachcited by 1
  34. 2023Back to Basics: A Simple Recipe for Improving Out-of-Domain Retrieval in Dense Encoderscited by 1
  35. 2024Leveraging Large Language Models for Structure Learning in Prompted Weak Supervisioncited by 1
  36. 2025LMAR: Language Model Augmented Retriever for Domain-specific Knowledge Indexingcited by 1
  37. 2026DALDALL: Data Augmentation for Lexical and Semantic Diverse in Legal Domain by leveraging LLM-Personacited by 1
  38. 2023Unsupervised Domain Adaption for Neural Information Retrievalcited by 0

50 most cited of 60 All 60 (JSON)