IR2: Information Regularization for Information Retrieval
Not on the map. Why?
Abstract (arXiv)
Effective information retrieval (IR) in settings with limited training data, particularly for complex queries, remains a challenging task. This paper introduces IR2, Information Regularization for Information Retrieval, a technique for reducing overfitting during synthetic data generation. This approach, representing a novel application of regularization techniques in synthetic data creation for IR, is tested on three recent IR tasks characterized by complex queries: DORIS-MAE, ArguAna, and WhatsThatBook. Experimental results indicate that our regularization techniques not only outperform previous synthetic query generation methods on the tasks considered but also reduce cost by up to 50%. Furthermore, this paper categorizes and explores three regularization methods at different stages of the query synthesis pipeline-input, prompt, and output-each offering varying degrees of performance improvement compared to models where no regularization is applied. This provides a systematic approach for optimizing synthetic data generation in data-limited, complex-query IR scenarios. All code, prompts and synthetic data are available at https://github.com/Info-Regularization/Information-Regularization.
Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 28 of its 51 references that are in the map. First posted on arXiv 25 Feb 2024.
Builds on
28 of 51 references.
References · 28
- 2019RoBERTa: A Robustly Optimized BERT Pretraining Approachcited by 13,266
- 2020A Simple Framework for Contrastive Learning of Visual Representationscited by 11,979
- 2022PaLM: Scaling Language Modeling with Pathwayscited by 5,304
- 2022Large Language Models are Zero-Shot Reasonerscited by 4,323
- 2021Finetuned Language Models Are Zero-Shot Learnerscited by 3,747
- 2021SimCSE: Simple Contrastive Learning of Sentence Embeddingscited by 2,168
- 2016MS MARCO: A Human Generated MAchine Reading COmprehension Datasetcited by 1,591
- 2021BEIR: A Heterogenous Benchmark for Zero-shot Evaluation of Information Retrieval Modelscited by 1,225
- 2022Text Embeddings by Weakly-Supervised Contrastive Pre-trainingcited by 1,216
- 2021Unsupervised Dense Information Retrieval with Contrastive Learningcited by 1,203
- 2023ChatGPT Outperforms Crowd-Workers for Text-Annotation Taskscited by 849
- 2020Approximate Nearest Neighbor Negative Contrastive Learning for Dense Text Retrievalcited by 846
16 more
- 2020Fact or Fiction: Verifying Scientific Claimscited by 546
- 2021ColBERTv2: Effective and Efficient Retrieval via Lightweight Late Interactioncited by 464
- 2020SPECTER: Document-level Representation Learning using Citation-informed Transformerscited by 426
- 2023Is ChatGPT Good at Search? Investigating Large Language Models as Re-Ranking Agentscited by 356
- 2022Promptagator: Few-shot Dense Retrieval From 8 Examplescited by 233
- 2022DiffCSE: Difference-based Contrastive Learning for Sentence Embeddingscited by 131
- 2021SPLADE v2: Sparse Lexical and Expansion Model for Information Retrievalcited by 126
- 2021GPL: Generative Pseudo Labeling for Unsupervised Domain Adaptation of Dense Retrievalcited by 119
- 2022SimLM: Pre-training with Representation Bottleneck for Dense Passage Retrievalcited by 90
- 2023Exploiting Asymmetry for Synthetic Training Data Generation: SynthIE and the Case of Information Extractioncited by 89
- 2023InPars-v2: Large Language Models as Efficient Dataset Generators for Information Retrievalcited by 73
- 2020Embedding-based Zero-shot Retrieval through Query Generationcited by 24
- 2023Decomposing Complex Queries for Tip-of-the-tongue Retrievalcited by 24
- 2023RankCSE: Unsupervised Sentence Representations Learning via Learning to Rankcited by 23
- 2023DORIS-MAE: Scientific Document Retrieval using Multi-level Aspect-based Queriescited by 11
- 2023Improving Content Retrievability in Search with Controllable Query Generationcited by 6
Built on by
Not cited yet.