# Using Pretrained Large Language Model with Prompt Engineering to Answer Biomedical Questions

arXiv: https://arxiv.org/abs/2407.06779 · HTML: https://mlsanity.com/paper/2407.06779/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- Builds on 4 of its 16 references that are in the map.
- First posted on arXiv 9 Jul 2024.

## Builds on (most cited first, 4 of 4)

- [LLaMA: Open and Efficient Foundation Language Models](https://mlsanity.com/paper/2302.13971/) (2023)
- [Mixtral of Experts](https://mlsanity.com/paper/2401.04088/) (2024)
- [A Prompt Pattern Catalog to Enhance Prompt Engineering with ChatGPT](https://mlsanity.com/paper/2302.11382/) (2023)
- [Overview of BioASQ 2023: The eleventh BioASQ challenge on Large-Scale Biomedical Semantic Indexing and Question Answering](https://mlsanity.com/paper/2307.05131/) (2023)

## Built on by (most cited first, 4 of 4)

- [Probabilistic Medical Predictions of Large Language Models](https://mlsanity.com/paper/2408.11316/) (2024)
- [Overview of BioASQ 2024: The twelfth BioASQ challenge on Large-Scale Biomedical Semantic Indexing and Question Answering](https://mlsanity.com/paper/2508.20532/) (2025)
- [Evaluating the Unseen Capabilities: How Many Theorems Do LLMs Know?](https://mlsanity.com/paper/2506.02058/) (2025)
- [AI-Generated Fall Data: Assessing LLMs and Diffusion Model for Wearable Fall Detection](https://mlsanity.com/paper/2505.04660/) (2025)

## Abstract (arXiv)

Our team participated in the BioASQ 2024 Task12b and Synergy tasks to build a system that can answer biomedical questions by retrieving relevant articles and snippets from the PubMed database and generating exact and ideal answers. We propose a two-level information retrieval and question-answering system based on pre-trained large language models (LLM), focused on LLM prompt engineering and response post-processing. We construct prompts with in-context few-shot examples and utilize post-processing techniques like resampling and malformed response detection. We compare the performance of various pre-trained LLM models on this challenge, including Mixtral, OpenAI GPT and Llama2. Our best-performing system achieved 0.14 MAP score on document retrieval, 0.05 MAP score on snippet retrieval, 0.96 F1 score for yes/no questions, 0.38 MRR score for factoid questions and 0.50 F1 score for list questions in Task 12b.