Two-Layer Retrieval-Augmented Generation Framework for Low-Resource Medical Question Answering Using Reddit Data: Proof-of-Concept Study
On the map: Retrieval-augmented generation → Medical and biomedical RAG
Abstract (arXiv)
The increasing use of social media to share lived and living experiences of substance use presents a unique opportunity to obtain information on side effects, use patterns, and opinions on novel psychoactive substances. However, due to the large volume of data, obtaining useful insights through natural language processing technologies such as large language models is challenging. This paper aims to develop a retrieval-augmented generation (RAG) architecture for medical question answering pertaining to clinicians' queries on emerging issues associated with health-related topics, using user-generated medical information on social media. We proposed a two-layer RAG framework for query-focused answer generation and evaluated a proof of concept for the framework in the context of query-focused summary generation from social media forums, focusing on emerging drug-related information. Our modular framework generates individual summaries followed by an aggregated summary to answer medical queries from large amounts of user-generated social media data in an efficient manner. We compared the performance of a quantized large language model (Nous-Hermes-2-7B-DPO), deployable in low-resource settings, with GPT-4. For this proof-of-concept study, we used user-generated data from Reddit to answer clinicians' questions on the use of xylazine and ketamine. Our framework achieves comparable median scores in terms of relevance, length, hallucination, coverage, and coherence when evaluated using GPT-4 and Nous-Hermes-2-7B-DPO, evaluated for 20 queries with 76 samples. There was no statistically significant difference between the two for coverage, coherence, relevance, length, and hallucination. A statistically significant difference was noted for the Coleman-Liau Index. Our RAG framework can effectively answer medical questions about targeted topics and can be deployed in resource-constrained settings.
Key facts. Cited by 5 papers in the map (data through 14 Oct 2026). Placed in Medical and biomedical RAG, one of 15 directions in Retrieval-augmented generation. Builds on 1 of its 8 references that are in the map. First posted on arXiv 29 May 2024.
Builds on
1 of 8 references.
References · 1
Built on by
Citing papers per year: 2025: 5.
5 papers.
- 2025A Systematic Review of Key Retrieval-Augmented Generation (RAG) Systems: Progress, Gaps, and Future Directionscited by 31
- 2025DoctorRAG: Medical RAG Fusing Knowledge with Patient Analogy through Textual Gradientscited by 5
- 2025Contradictions in Context: Challenges for Retrieval-Augmented Generation in Healthcarecited by 1
- 2025A Multi-Agent Approach to Neurological Clinical Reasoningcited by 0
- 2025Mentalic Net: Development of RAG-based Conversational AI and Evaluation Framework for Mental Health Supportcited by 0
- 2025Contradictions in Context: Challenges for Retrieval-Augmented Generation in Healthcare10 Nov 2025
- 2025Mentalic Net: Development of RAG-based Conversational AI and Evaluation Framework for Mental Health Support27 Aug 2025
- 2025A Multi-Agent Approach to Neurological Clinical Reasoning10 Aug 2025
- 2025A Systematic Review of Key Retrieval-Augmented Generation (RAG) Systems: Progress, Gaps, and Future Directions25 Jul 2025
- 2025DoctorRAG: Medical RAG Fusing Knowledge with Patient Analogy through Textual Gradients26 May 2025
Similar, not cited
Close in wording, no citation either way
- 2024The Geometry of Queries: Query-Based Innovations in Retrieval-Augmented Generation for Healthcare QA
- 2023MKRAG: Medical Knowledge Retrieval Augmented Generation for Medical Question Answering
- 2024Comprehensive and Practical Evaluation of Retrieval-Augmented Generation Systems for Medical Question Answering
- 2026A Systematic Study of Retrieval Pipeline Design for Retrieval-Augmented Medical Question Answering
- 2025Optimizing Medical Question-Answering Systems: A Comparative Study of Fine-Tuned and Zero-Shot Large Language Models with RAG Framework
- 2025Retrieval-Augmented Generation in Medicine: A Scoping Review of Technical Implementations, Clinical Applications, and Ethical Considerations
- 2024Rationale-Guided Retrieval Augmented Generation for Medical Question Answering
- 2026Healthier LLMs: Retrieval-Augmented Generation for Public Health Question Answering