mlsanityData to 14 Oct 2026

Beyond Retrieval: Ensembling Cross-Encoders and GPT Rerankers with LLMs for Biomedical QA

arXiv 2507.05577 ↗First posted 8 Jul 2025cs.IR cs.CL cs.LGcited by 2

Not on the map. Why?

Abstract (arXiv)

Biomedical semantic question answering rooted in information retrieval can play a crucial role in keeping up to date with vast, rapidly evolving and ever-growing biomedical literature. A robust system can help researchers, healthcare professionals and even layman users access relevant knowledge grounded in evidence. The BioASQ 2025 Task13b Challenge serves as an important benchmark, offering a competitive platform for advancement of this space. This paper presents the methodologies and results from our participation in this challenge where we built a Retrieval-Augmented Generation (RAG) system that can answer biomedical questions by retrieving relevant PubMed documents and snippets to generate answers. For the retrieval task, we generated dense embeddings from biomedical articles for initial retrieval, and applied an ensemble of finetuned cross-encoders and large language models (LLMs) for re-ranking to identify top relevant documents. Our solution achieved an MAP@10 of 0.1581, placing 10th on the leaderboard for the retrieval task. For answer generation, we employed few-shot prompting of instruction-tuned LLMs. Our system achieved macro-F1 score of 0.95 for yes/no questions (rank 12), Mean Reciprocal Rank (MRR) of 0.64 for factoid questions (rank 1), mean-F1 score of 0.63 for list questions (rank 5), and ROUGE-SU4 F1 score of 0.29 for ideal answers (rank 11).

Key facts. Cited by 2 papers in the map (data through 14 Oct 2026). Builds on 19 of its 27 references that are in the map. First posted on arXiv 8 Jul 2025.

Builds on

19 of 27 references.

References · 19

  1. 2020Language Models are Few-Shot Learnerscited by 32,419
  2. 2023GPT-4 Technical Reportcited by 18,396
  3. 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,459
  4. 2020Retrieval-Augmented Generation for Knowledge-Intensive NLP Taskscited by 9,297
  5. 2019Sentence-BERT: Sentence Embeddings using Siamese BERT-Networkscited by 9,170
  6. 2017Billion-scale similarity search with GPUscited by 2,497
  7. 2019BioBERT: a pre-trained biomedical language representation model for biomedical text miningcited by 1,947
  8. 2020MiniLM: Deep Self-Attention Distillation for Task-Agnostic Compression of Pre-Trained Transformerscited by 1,463
  9. 2021BEIR: A Heterogenous Benchmark for Zero-shot Evaluation of Information Retrieval Modelscited by 1,252
  10. 2020Leveraging Passage Retrieval with Generative Models for Open Domain Question Answeringcited by 1,210
  11. 2020Domain-Specific Language Model Pretraining for Biomedical Natural Language Processingcited by 1,009
  12. 2016Efficient and robust approximate nearest neighbor search using Hierarchical Navigable Small World graphscited by 861
7 more
  1. 2020AdapterFusion: Non-Destructive Task Composition for Transfer Learningcited by 788
  2. 2019Passage Re-ranking with BERTcited by 645
  3. 2022BioGPT: Generative Pre-trained Transformer for Biomedical Text Generation and Miningcited by 538
  4. 2023Zero-Shot Listwise Document Reranking with a Large Language Modelcited by 82
  5. 2023RankVicuna: Zero-Shot Listwise Document Reranking with Open-Source Large Language Modelscited by 70
  6. 2022In Defense of Cross-Encoders for Zero-Shot Retrievalcited by 21
  7. 2024Can Open-Source LLMs Compete with Commercial Models? Exploring the Few-Shot Performance of Current GPT Models in Biomedical Taskscited by 6

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 1; 2026: 1.

2 papers.

  1. 2025Overview of BioASQ 2025: The Thirteenth BioASQ Challenge on Large-Scale Biomedical Semantic Indexing and Question Answeringcited by 11
  2. 2026Self-MedRAG: a Self-Reflective Hybrid Retrieval-Augmented Generation Framework for Reliable Medical Question Answeringcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025RAG-BioQA: A Retrieval-Augmented Generation Framework for Long-Form Biomedical Question Answering
  2. 2026Benchmarking Retrieval Strategies for Biomedical Retrieval-Augmented Generation: A Controlled Empirical Study
  3. 2025Efficient and Reproducible Biomedical Question Answering using Retrieval Augmented Generation
  4. 2025Harnessing Collective Intelligence of LLMs for Robust Biomedical QA: A Multi-Model Approach
  5. 2025Biomedical Literature Q&A System Using Retrieval-Augmented Generation (RAG)
  6. 2024How do you know that? Teaching Generative Language Models to Reference Answers to Biomedical Questions
  7. 2023MKRAG: Medical Knowledge Retrieval Augmented Generation for Medical Question Answering
  8. 2026A Systematic Study of Retrieval Pipeline Design for Retrieval-Augmented Medical Question Answering