RE-AdaptIR: Improving Information Retrieval through Reverse Engineered Adaptation
Not on the map. Why?
Abstract (arXiv)
Large language models (LLMs) fine-tuned for text-retrieval have demonstrated state-of-the-art results across several information retrieval (IR) benchmarks. However, supervised training for improving these models requires numerous labeled examples, which are generally unavailable or expensive to acquire. In this work, we explore the effectiveness of extending reverse engineered adaptation to the context of information retrieval (RE-AdaptIR). We use RE-AdaptIR to improve LLM-based IR models using only unlabeled data. We demonstrate improved performance both in training domains as well as zero-shot in domains where the models have seen no queries. We analyze performance changes in various fine-tuning scenarios and offer findings of immediate use to practitioners.
Key facts. Cited by 1 paper in the map (data through 13 Oct 2026). Builds on 22 of its 33 references that are in the map. First posted on arXiv 20 Jun 2024.
Builds on
22 of 33 references.
References · 22
- 2022Training language models to follow instructions with human feedbackcited by 15,856
- 2021LoRA: Low-Rank Adaptation of Large Language Modelscited by 14,231
- 2023Llama 2: Open Foundation and Fine-Tuned Chat Modelscited by 12,979
- 2020Retrieval-Augmented Generation for Knowledge-Intensive NLP Taskscited by 9,221
- 2019Sentence-BERT: Sentence Embeddings using Siamese BERT-Networkscited by 9,107
- 2018Representation Learning with Contrastive Predictive Codingcited by 7,712
- 2019Parameter-Efficient Transfer Learning for NLPcited by 4,213
- 2020Dense Passage Retrieval for Open-Domain Question Answeringcited by 3,797
- 2021Finetuned Language Models Are Zero-Shot Learnerscited by 3,765
- 2018HotpotQA: A Dataset for Diverse, Explainable Multi-hop Question Answeringcited by 3,716
- 2016MS MARCO: A Human Generated MAchine Reading COmprehension Datasetcited by 1,600
- 2018FEVER: a large-scale dataset for Fact Extraction and VERificationcited by 1,368
10 more
- 2021BEIR: A Heterogenous Benchmark for Zero-shot Evaluation of Information Retrieval Modelscited by 1,237
- 2021Unsupervised Dense Information Retrieval with Contrastive Learningcited by 1,205
- 2020ColBERT: Efficient and Effective Passage Search via Contextualized Late Interaction over BERTcited by 1,169
- 2022Editing Models with Task Arithmeticcited by 1,101
- 2021Cross-Task Generalization via Natural Language Crowdsourcing Instructionscited by 697
- 2023Improving Text Embeddings with Large Language Modelscited by 297
- 2023Fine-Tuning LLaMA for Multi-Stage Text Retrievalcited by 271
- 2023Understanding Catastrophic Forgetting in Language Models via Implicit Inferencecited by 141
- 2024FollowIR: Evaluating and Teaching Information Retrieval Models to Follow Instructionscited by 68
- 2024RE-Adapt: Reverse Engineered Adaptation of Large Language Modelscited by 9
Built on by
Citing papers per year: 2026: 1.
1 paper.