mlsanityData to 7 Oct 2026

BMRetriever: Tuning Large Language Models as Better Biomedical Text Retrievers

arXiv 2404.18443 ↗First posted 29 Apr 2024cs.CL cs.AI cs.IRcited by 38

Not on the map. Why?

Abstract (arXiv)

Developing effective biomedical retrieval models is important for excelling at knowledge-intensive biomedical tasks but still challenging due to the deficiency of sufficient publicly annotated biomedical data and computational resources. We present BMRetriever, a series of dense retrievers for enhancing biomedical retrieval via unsupervised pre-training on large biomedical corpora, followed by instruction fine-tuning on a combination of labeled datasets and synthetic pairs. Experiments on 5 biomedical tasks across 11 datasets verify BMRetriever's efficacy on various biomedical applications. BMRetriever also exhibits strong parameter efficiency, with the 410M variant outperforming baselines up to 11.7 times larger, and the 2B variant matching the performance of models with over 5B parameters. The training data and model checkpoints are released at urlhttps://huggingface.co/BMRetriever to ensure transparency, reproducibility, and application to new domains.

Key facts. Cited by 38 papers in the map (data through 7 Oct 2026). Builds on 60 of its 98 references that are in the map. First posted on arXiv 29 Apr 2024.

Builds on

60 of 98 references.

References · 60

  1. 2020Language Models are Few-Shot Learnerscited by 32,199
  2. 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,491
  3. 2020Retrieval-Augmented Generation for Knowledge-Intensive NLP Taskscited by 9,160
  4. 2020Dense Passage Retrieval for Open-Domain Question Answeringcited by 3,779
  5. 2018HotpotQA: A Dataset for Diverse, Explainable Multi-hop Question Answeringcited by 3,686
  6. 2015A large annotated corpus for learning natural language inferencecited by 1,837
  7. 2023Self-RAG: Learning to Retrieve, Generate, and Critique through Self-Reflectioncited by 1,800
  8. 2023Pythia: A Suite for Analyzing Large Language Models Across Training and Scalingcited by 1,734
  9. 2016MS MARCO: A Human Generated MAchine Reading COmprehension Datasetcited by 1,591
  10. 2020Don't Stop Pretraining: Adapt Language Models to Domains and Taskscited by 1,548
  11. 2018FEVER: a large-scale dataset for Fact Extraction and VERificationcited by 1,362
  12. 2020What Disease does this Patient Have? A Large-scale Open Domain Question Answering Dataset from Medical Examscited by 1,324
38 more
  1. 2019PubMedQA: A Dataset for Biomedical Research Question Answeringcited by 1,271
  2. 2021BEIR: A Heterogenous Benchmark for Zero-shot Evaluation of Information Retrieval Modelscited by 1,225
  3. 2022Text Embeddings by Weakly-Supervised Contrastive Pre-trainingcited by 1,216
  4. 2021Unsupervised Dense Information Retrieval with Contrastive Learningcited by 1,203
  5. 2024M3-Embedding: Multi-Linguality, Multi-Functionality, Multi-Granularity Text Embeddings Through Self-Knowledge Distillationcited by 1,050
  6. 2020Approximate Nearest Neighbor Negative Contrastive Learning for Dense Text Retrievalcited by 846
  7. 2019Latent Retrieval for Weakly Supervised Open Domain Question Answeringcited by 691
  8. 2019ELI5: Long Form Question Answeringcited by 567
  9. 2020Fact or Fiction: Verifying Scientific Claimscited by 546
  10. 2019CCNet: Extracting High Quality Monolingual Datasets from Web Crawl Datacited by 544
  11. 2022BioGPT: Generative Pre-trained Transformer for Biomedical Text Generation and Miningcited by 531
  12. 2021Large Dual Encoders Are Generalizable Retrieverscited by 454
  13. 2020SPECTER: Document-level Representation Learning using Citation-informed Transformerscited by 426
  14. 2020RocketQA: An Optimized Training Approach to Dense Passage Retrieval for Open-Domain Question Answeringcited by 410
  15. 2024LLM2Vec: Large Language Models Are Secretly Powerful Text Encoderscited by 396
  16. 2022Text and Code Embeddings by Contrastive Pre-Trainingcited by 358
  17. 2023NLP Evaluation in trouble: On the Need to Measure LLM Data Contamination for each Benchmarkcited by 339
  18. 2022One Embedder, Any Task: Instruction-Finetuned Text Embeddingscited by 329
  19. 2024Benchmarking Retrieval-Augmented Generation for Medicinecited by 326
  20. 2023ChatDoctor: A Medical Chat Model Fine-Tuned on a Large Language Model Meta-AI (LLaMA) Using Medical Domain Knowledgecited by 324
  21. 2024BioMistral: A Collection of Open-Source Pretrained Large Language Models for Medical Domainscited by 299
  22. 2023Fine-Tuning LLaMA for Multi-Stage Text Retrievalcited by 271
  23. 2024RAFT: Adapting Language Model to Domain Specific RAGcited by 258
  24. 2020Self-Alignment Pretraining for Biomedical Entity Representationscited by 233
  25. 2022Promptagator: Few-shot Dense Retrieval From 8 Examplescited by 233
  26. 2024Generative Representational Instruction Tuningcited by 227
  27. 2022SGPT: GPT Sentence Embeddings for Semantic Searchcited by 204
  28. 2021Unsupervised Corpus Aware Language Model Pre-training for Dense Passage Retrievalcited by 203
  29. 2023MedCPT: Contrastive Pre-trained Transformers with Large-scale PubMed Search Logs for Zero-shot Biomedical Information Retrievalcited by 182
  30. 2020CORD-19: The COVID-19 Open Research Datasetcited by 179
  31. 2020MedDialog: Two Large-scale Medical Dialogue Datasetscited by 157
  32. 2023Med-HALT: Medical Domain Hallucination Test for Large Language Modelscited by 156
  33. 2019Synthetic QA Corpora Generation with Roundtrip Consistencycited by 143
  34. 2023PaperQA: Retrieval-Augmented Generative Agent for Scientific Researchcited by 143
  35. 2022SciRepEval: A Multi-Format Benchmark for Scientific Document Representationscited by 124
  36. 2019Learning Dense Representations for Entity Retrievalcited by 122
  37. 2021GPL: Generative Pseudo Labeling for Unsupervised Domain Adaptation of Dense Retrievalcited by 119
  38. 2022RetroMAE: Pre-Training Retrieval-oriented Language Models Via Masked Auto-Encodercited by 118

50 most cited of 60 All 60 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 10; 2025: 20; 2026: 8.

38 papers.

Sort:

By area:

  1. 2024A Comprehensive Survey of Scientific Large Language Models and Their Applications in Scientific Discoverycited by 96
  2. 2024HYDRA: Model Factorization Framework for Black-Box LLM Personalizationcited by 59
  3. 2024Visual Haystacks: A Vision-Centric Needle-In-A-Haystack Benchmarkcited by 53
  4. 2024SimRAG: Self-Improving Retrieval-Augmented Generation for Adapting Large Language Models to Specialized Domainscited by 28
  5. 2024OG-RAG: Ontology-Grounded Retrieval-Augmented Generation For Large Language Modelscited by 23
  6. 2025jina-reranker-v3: Last but Not Late Interaction for Listwise Document Rerankingcited by 23
  7. 2024Efficient Knowledge Injection in LLMs via Self-Distillationcited by 22
  8. 2025Kanana: Compute-efficient Bilingual Language Modelscited by 19
  9. 2025Collab-RAG: Boosting Retrieval-Augmented Generation for Complex Question Answering via White-Box and Black-Box LLM Collaborationcited by 19
  10. 2024Towards Reliable Medical Question Answering: Techniques and Challenges in Mitigating Hallucinations in Language Modelscited by 15
  11. 2025Towards Omni-RAG: Comprehensive Retrieval-Augmented Generation for Large Language Models in Medical Applicationscited by 13
  12. 2024RAMIE: Retrieval-Augmented Multi-task Information Extraction with Large Language Models on Dietary Supplementscited by 11
26 more
  1. 2025End-to-End Agentic RAG System Training for Traceable Diagnostic Reasoningcited by 10
  2. 2024AutoMIR: Effective Zero-Shot Medical Information Retrieval without Relevance Labelscited by 8
  3. 2025RAGRouter: Learning to Route Queries to Multiple Retrieval-Augmented Language Modelscited by 8
  4. 2025Decoding Dense Embeddings: Sparse Autoencoders for Interpreting and Discretizing Dense Retrievalcited by 8
  5. 2025A Collection of Innovations in Medical AI for patient records in 2024cited by 7
  6. 2025FailureSensorIQ: A Multi-Choice QA Dataset for Understanding Sensor Relationships and Failure Modescited by 6
  7. 2024Intermediate Distillation: Data-Efficient Distillation from Black-Box LLMs for Information Retrievalcited by 3
  8. 2025DisastIR: A Comprehensive Information Retrieval Benchmark for Disaster Managementcited by 3
  9. 2025RAG in the Wild: On the (In)effectiveness of LLMs with Mixture-of-Knowledge Retrieval Augmentationcited by 3
  10. 2026Tracing the Data Trail: A Survey of Data Provenance, Transparency and Traceability in LLMscited by 3
  11. 2025DMRetriever: A Family of Models for Improved Text Retrieval in Disaster Managementcited by 2
  12. 2025Data-Efficient Biomedical In-Context Learning: A Diversity-Enhanced Submodular Perspectivecited by 1
  13. 2025Exploring the Capabilities of Large Language Model Encoders for Image-Text Retrieval in Chest X-rayscited by 1
  14. 2025GRIL: Knowledge Graph Retrieval-Integrated Learning with Large Language Modelscited by 1
  15. 2026REPAIR: Resolving Long-Tail Confusion in Scientific Retrievers via Fact-Verified Iterative Refinementcited by 1
  16. 2025Empowering Large Language Models in Wireless Communication: A Novel Dataset and Fine-Tuning Frameworkcited by 0
  17. 2025DR.EHR: Dense Retrieval for Electronic Health Record with Knowledge Injection and Synthetic Datacited by 0
  18. 2025Ontology-Guided Query Expansion for Biomedical Document Retrieval using Large Language Modelscited by 0
  19. 2025MedSEBA: Synthesizing Evidence-Based Answers Grounded in Evolving Medical Literaturecited by 0
  20. 2025Investigating LLM Capabilities on Long Context Comprehension for Medical Question Answeringcited by 0
  21. 2026Modular Expert Merging for Biomedical Retrievalcited by 0
  22. 2026Asking the Right Questions: Ontology-Grounded Interpretable Embeddings for Biomedical Textcited by 0
  23. 2026From Exposure to Internalization: Dual-Stream Calibration for In-context Clinical Reasoningcited by 0
  24. 2026BioHiCL: Hierarchical Multi-Label Contrastive Learning for Biomedical Retrieval with MeSH Labelscited by 0
  25. 2026When Cases Get Rare: A Retrieval Benchmark for Off-Guideline Clinical Question Answeringcited by 0
  26. 2026ACE: Adapter Consolidation across Experts for Parameter-Efficient Fine-Tuning of MoE LLMscited by 0