REPAIR: Resolving Long-Tail Confusion in Scientific Retrievers via Fact-Verified Iterative Refinement
Not on the map. Why?
Abstract (arXiv)
Precise retrieval of scientific information is fundamentally constrained by long-tailed concepts and high fact-sensitivity of scientific corpora. These challenges often limit the effectiveness of dense retrievers and hallucination-prone LLM augmentation. To address this, we present REPAIR, a self-evolving data augmentation framework for scientific dense retrievers. REPAIR iteratively synthesizes training data to address knowledge gaps by cycling through diagnosis of long-tail concepts, API-guided evidence expansion, and differentiation via hard negative mining. This process effectively grounds retrieval in factual reality to resolve fine-grained distinctions. Extensive experiments demonstrate that REPAIR significantly outperforms 19 strong baselines on nine materials science and biomedical benchmarks. Our work highlights that diagnosing and factually augmenting data to long-tail deficits is essential for robust scientific retrieval.
Key facts. Cited by 1 paper in the map (data through 14 Oct 2026). Builds on 41 of its 75 references that are in the map. First posted on arXiv 16 Sep 2026.
Builds on
41 of 75 references.
References · 41
- 2017Attention Is All You Needcited by 60,361
- 2020Language Models are Few-Shot Learnerscited by 32,419
- 2023Llama 2: Open Foundation and Fine-Tuned Chat Modelscited by 13,028
- 2019Unsupervised Cross-lingual Representation Learning at Scalecited by 3,935
- 2016Pointer Sentinel Mixture Modelscited by 3,096
- 2023Pythia: A Suite for Analyzing Large Language Models Across Training and Scalingcited by 1,786
- 2016MS MARCO: A Human Generated MAchine Reading COmprehension Datasetcited by 1,605
- 2019SciBERT: A Pretrained Language Model for Scientific Textcited by 1,414
- 2022Text Embeddings by Weakly-Supervised Contrastive Pre-trainingcited by 1,253
- 2021BEIR: A Heterogenous Benchmark for Zero-shot Evaluation of Information Retrieval Modelscited by 1,252
- 2021Unsupervised Dense Information Retrieval with Contrastive Learningcited by 1,216
- 2024M3-Embedding: Multi-Linguality, Multi-Functionality, Multi-Granularity Text Embeddings Through Self-Knowledge Distillationcited by 1,073
29 more
- 2020Domain-Specific Language Model Pretraining for Biomedical Natural Language Processingcited by 1,009
- 2017Crowdsourcing Multiple Choice Science Questionscited by 765
- 2020Fact or Fiction: Verifying Scientific Claimscited by 554
- 2021Large Dual Encoders Are Generalizable Retrieverscited by 456
- 2020SPECTER: Document-level Representation Learning using Citation-informed Transformerscited by 430
- 2024LLM2Vec: Large Language Models Are Secretly Powerful Text Encoderscited by 404
- 2022Text and Code Embeddings by Contrastive Pre-Trainingcited by 361
- 2022One Embedder, Any Task: Instruction-Finetuned Text Embeddingscited by 333
- 2024Benchmarking Retrieval-Augmented Generation for Medicinecited by 331
- 2024BioMistral: A Collection of Open-Source Pretrained Large Language Models for Medical Domainscited by 301
- 2023Fine-Tuning LLaMA for Multi-Stage Text Retrievalcited by 272
- 2022SGPT: GPT Sentence Embeddings for Semantic Searchcited by 204
- 2023MedCPT: Contrastive Pre-trained Transformers with Large-scale PubMed Search Logs for Zero-shot Biomedical Information Retrievalcited by 185
- 2020CORD-19: The COVID-19 Open Research Datasetcited by 180
- 2020MedDialog: Two Large-scale Medical Dialogue Datasetscited by 161
- 2023Med-HALT: Medical Domain Hallucination Test for Large Language Modelscited by 157
- 2022SciRepEval: A Multi-Format Benchmark for Scientific Document Representationscited by 129
- 2023How to Train Your DRAGON: Diverse Augmentation Towards Generalizable Dense Retrievalcited by 118
- 2019Efficient Parameter-free Clustering Using First Neighbor Relationscited by 97
- 2023Llama2Vec: Unsupervised Adaptation of Large Language Models for Dense Retrievalcited by 78
- 2024HoneyComb: A Flexible LLM-Based Agent System for Materials Sciencecited by 57
- 2024Rationale-Guided Retrieval Augmented Generation for Medical Question Answeringcited by 41
- 2024Promptriever: Instruction-Trained Retrievers Can Be Prompted Like Language Modelscited by 40
- 2024BMRetriever: Tuning Large Language Models as Better Biomedical Text Retrieverscited by 38
- 2025DRAMA: Diverse Augmentation from Large Language Models to Smaller Dense Retrieverscited by 19
- 2023Pre-training Multi-task Contrastive Learning Models for Scientific Literature Understandingcited by 16
- 2023DORIS-MAE: Scientific Document Retrieval using Multi-level Aspect-based Queriescited by 11
- 2024MELT: Materials-aware Continued Pre-training for Language Model Adaptation to Materials Sciencecited by 5
- 2025Incorporating Domain Knowledge into Materials Tokenizationcited by 2
Built on by
Citing papers per year: 2026: 1.
1 paper.