# Improving Biomedical Information Retrieval with Neural Retrievers

arXiv: https://arxiv.org/abs/2201.07745 · HTML: https://mlsanity.com/paper/2201.07745/

## Key facts

- Cited by 20 papers in the map (data through 14 Oct 2026).
- Builds on 30 of its 52 references that are in the map.
- First posted on arXiv 19 Jan 2022.

## Builds on (most cited first, 30 of 30)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer](https://mlsanity.com/paper/1910.10683/) (2019)
- [Dense Passage Retrieval for Open-Domain Question Answering](https://mlsanity.com/paper/2004.04906/) (2020)
- [Sequence to Sequence Learning with Neural Networks](https://mlsanity.com/paper/1409.3215/) (2014)
- [TriviaQA: A Large Scale Distantly Supervised Challenge Dataset for Reading Comprehension](https://mlsanity.com/paper/1705.03551/) (2017)
- [REALM: Retrieval-Augmented Language Model Pre-Training](https://mlsanity.com/paper/2002.08909/) (2020)
- [BioBERT: a pre-trained biomedical language representation model for biomedical text mining](https://mlsanity.com/paper/1901.08746/) (2019)
- [Google's Neural Machine Translation System: Bridging the Gap between Human and Machine Translation](https://mlsanity.com/paper/1609.08144/) (2016)
- [BEIR: A Heterogenous Benchmark for Zero-shot Evaluation of Information Retrieval Models](https://mlsanity.com/paper/2104.08663/) (2021)
- [ColBERT: Efficient and Effective Passage Search via Contextualized Late Interaction over BERT](https://mlsanity.com/paper/2004.12832/) (2020)
- [Latent Retrieval for Weakly Supervised Open Domain Question Answering](https://mlsanity.com/paper/1906.00300/) (2019)
- [Unified Language Model Pre-training for Natural Language Understanding and Generation](https://mlsanity.com/paper/1905.03197/) (2019)
- [Passage Re-ranking with BERT](https://mlsanity.com/paper/1901.04085/) (2019)
- [Poly-encoders: Transformer Architectures and Pre-training Strategies for Fast and Accurate Multi-sentence Scoring](https://mlsanity.com/paper/1905.01969/) (2019)
- [Pre-training Tasks for Embedding-based Large-scale Retrieval](https://mlsanity.com/paper/2002.03932/) (2020)
- [End-to-End Open-Domain Question Answering with BERTserini](https://mlsanity.com/paper/1902.01718/) (2019)
- [Synthetic QA Corpora Generation with Roundtrip Consistency](https://mlsanity.com/paper/1906.05416/) (2019)
- [Simple Entity-Centric Questions Challenge Dense Retrievers](https://mlsanity.com/paper/2109.08535/) (2021)
- [Question and Answer Test-Train Overlap in Open-Domain Question Answering Datasets](https://mlsanity.com/paper/2008.02637/) (2020)
- [Asymmetric LSH (ALSH) for Sublinear Time Maximum Inner Product Search (MIPS)](https://mlsanity.com/paper/1405.5869/) (2014)
- [CEDR: Contextualized Embeddings for Document Ranking](https://mlsanity.com/paper/1904.07094/) (2019)
- [Pyserini: An Easy-to-Use Python Toolkit to Support Replicable IR Research with Sparse and Dense Representations](https://mlsanity.com/paper/2102.10073/) (2021)
- [Unsupervised Question Answering by Cloze Translation](https://mlsanity.com/paper/1906.04980/) (2019)
- [Harvesting Paragraph-Level Question-Answer Pairs from Wikipedia](https://mlsanity.com/paper/1805.05942/) (2018)
- [Visual Question Generation as Dual Task of Visual Question Answering](https://mlsanity.com/paper/1709.07192/) (2017)
- [Information Maximizing Visual Question Generation](https://mlsanity.com/paper/1903.11207/) (2019)
- [Template-Based Question Generation from Retrieved Sentences for Improved Unsupervised Question Answering](https://mlsanity.com/paper/2004.11892/) (2020)
- [WeaQA: Weak Supervision via Captions for Visual Question Answering](https://mlsanity.com/paper/2012.02356/) (2020)
- [Self-Supervised Test-Time Learning for Reading Comprehension](https://mlsanity.com/paper/2103.11263/) (2021)
- [Factorising Meaning and Form for Intent-Preserving Paraphrasing](https://mlsanity.com/paper/2105.15053/) (2021)

## Built on by (most cited first, 20 of 20)

- [Task-aware Retrieval with Instructions](https://mlsanity.com/paper/2211.09260/) (2022)
- [Bridging the Gap Between Indexing and Retrieval for Differentiable Search Index with Query Generation](https://mlsanity.com/paper/2206.10128/) (2022)
- [BMRetriever: Tuning Large Language Models as Better Biomedical Text Retrievers](https://mlsanity.com/paper/2404.18443/) (2024)
- [End-to-end Knowledge Retrieval with Multi-modal Queries](https://mlsanity.com/paper/2306.00424/) (2023)
- [Retrieval augmentation of large language models for lay language generation](https://mlsanity.com/paper/2211.03818/) (2022)
- [In-BoXBART: Get Instructions into Biomedical Multi-Task Learning](https://mlsanity.com/paper/2204.07600/) (2022)
- [BioTABQA: Instruction Learning for Biomedical Table Question Answering](https://mlsanity.com/paper/2207.02419/) (2022)
- [Assessing Empathy in Large Language Models with Real-World Physician-Patient Interactions](https://mlsanity.com/paper/2405.16402/) (2024)
- [Choose Your QA Model Wisely: A Systematic Study of Generative and Extractive Readers for Question Answering](https://mlsanity.com/paper/2203.07522/) (2022)
- [MDDial: A Multi-turn Differential Diagnosis Dialogue Dataset with Reliability Evaluation](https://mlsanity.com/paper/2308.08147/) (2023)
- [Toward Reliable Scientific Hypothesis Generation: Evaluating Truthfulness and Hallucination in Large Language Models](https://mlsanity.com/paper/2505.14599/) (2025)
- [A Study on the Efficiency and Generalization of Light Hybrid Retrievers](https://mlsanity.com/paper/2210.01371/) (2022)
- [Medical Dialogue: A Survey of Categories, Methods, Evaluation and Challenges](https://mlsanity.com/paper/2405.10630/) (2024)
- [Neural Retriever and Go Beyond: A Thesis Proposal](https://mlsanity.com/paper/2205.16005/) (2022)
- [Diversifying Knowledge Enhancement of Biomedical Language Models using Adapter Modules and Knowledge Graphs](https://mlsanity.com/paper/2312.13881/) (2023)
- [Empowering Language Model with Guided Knowledge Fusion for Biomedical Document Re-ranking](https://mlsanity.com/paper/2305.04344/) (2023)
- [Cohort Retrieval using Dense Passage Retrieval](https://mlsanity.com/paper/2507.01049/) (2025)
- [Investigating LLM Capabilities on Long Context Comprehension for Medical Question Answering](https://mlsanity.com/paper/2510.18691/) (2025)
- [Divide-Prompt-Refine: a Training-Free, Structure-Aware Framework for Biomedical Abstract Generation](https://mlsanity.com/paper/2605.20628/) (2026)
- [GrocLM: Grocery Category Recommendation in E-Commerce with Large Language Models](https://mlsanity.com/paper/2607.24764/) (2026)

## Abstract (arXiv)

Information retrieval (IR) is essential in search engines and dialogue systems as well as natural language processing tasks such as open-domain question answering. IR serve an important function in the biomedical domain, where content and sources of scientific knowledge may evolve rapidly. Although neural retrievers have surpassed traditional IR approaches such as TF-IDF and BM25 in standard open-domain question answering tasks, they are still found lacking in the biomedical domain. In this paper, we seek to improve information retrieval (IR) using neural retrievers (NR) in the biomedical domain, and achieve this goal using a three-pronged approach. First, to tackle the relative lack of data in the biomedical domain, we propose a template-based question generation method that can be leveraged to train neural retriever models. Second, we develop two novel pre-training tasks that are closely aligned to the downstream task of information retrieval. Third, we introduce the ``Poly-DPR'' model which encodes each context into multiple context vectors. Extensive experiments and analysis on the BioASQ challenge suggest that our proposed method leads to large gains over existing neural approaches and beats BM25 in the small-corpus setting. We show that BM25 and our method can complement each other, and a simple hybrid model leads to further gains in the large corpus setting.