Exploring the Role of Knowledge Graph-Based RAG in Japanese Medical Question Answering with Small-Scale LLMs
On the map: Retrieval-augmented generation → Medical and biomedical RAG
Abstract (arXiv)
Large language models (LLMs) perform well in medical QA, but their effectiveness in Japanese contexts is limited due to privacy constraints that prevent the use of commercial models like GPT-4 in clinical settings. As a result, recent efforts focus on instruction-tuning open-source LLMs, though the potential of combining them with retrieval-augmented generation (RAG) remains underexplored. To bridge this gap, we are the first to explore a knowledge graph-based (KG) RAG framework for Japanese medical QA small-scale open-source LLMs. Experimental results show that KG-based RAG has only a limited impact on Japanese medical QA using small-scale open-source LLMs. Further case studies reveal that the effectiveness of the RAG is sensitive to the quality and relevance of the external retrieved content. These findings offer valuable insights into the challenges and potential of applying RAG in Japanese medical QA, while also serving as a reference for other low-resource languages.
Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in Medical and biomedical RAG, one of 15 directions in Retrieval-augmented generation. Builds on 17 of its 29 references that are in the map. First posted on arXiv 15 Apr 2025.
Builds on
17 of 29 references.
Closest ancestors · 2
Same direction, newest first
- 2024Benchmarking Retrieval-Augmented Generation for Medicine20 Feb
- 2023MKRAG: Medical Knowledge Retrieval Augmented Generation for Medical Question Answering27 Sep
Other references · 15
- 2023LLaMA: Open and Efficient Foundation Language Modelscited by 14,132
- 2019BERTScore: Evaluating Text Generation with BERTcited by 5,512
- 2024GPT-4o System Cardcited by 4,588
- 2024From Local to Global: A Graph RAG Approach to Query-Focused Summarizationcited by 1,453
- 2023Capabilities of GPT-4 on Medical Challenge Problemscited by 638
- 2023How Language Model Hallucinations Can Snowballcited by 311
- 2023Sources of Hallucination by Large Language Models on Inference Taskscited by 169
- 2023ExpertQA: Expert-Curated Questions and Attributed Answerscited by 97
- 2025MMLU-ProX: A Multilingual Benchmark for Advanced Large Language Model Evaluationcited by 79
- 2023Evaluating GPT-4 and ChatGPT on Japanese Medical Licensing Examinationscited by 63
- 2024KG-Rank: Enhancing Large Language Models for Medical QA with Knowledge Graphs and Ranking Techniquescited by 37
- 2024LLM-MedQA: Enhancing Medical Question Answering through Case Studies in Large Language Modelscited by 32
3 more
- 2025GraphCheck: Breaking Long-Term Text Barriers with Extracted Knowledge Graph-Powered Fact-Checkingcited by 12
- 2024Multilingual Hallucination Gaps in Large Language Modelscited by 7
- 2025MKG-Rank: Enhancing Large Language Models with Knowledge Graph for Multilingual Medical Question Answeringcited by 3
Built on by
Not cited yet.
Similar, not cited
Close in wording, no citation either way
- 2024Comprehensive and Practical Evaluation of Retrieval-Augmented Generation Systems for Medical Question Answering
- 2025Optimizing Medical Question-Answering Systems: A Comparative Study of Fine-Tuned and Zero-Shot Large Language Models with RAG Framework
- 2024Improving Retrieval-Augmented Generation in Medicine with Iterative Follow-up Questions
- 2024The Geometry of Queries: Query-Based Innovations in Retrieval-Augmented Generation for Healthcare QA
- 2026A Systematic Study of Retrieval Pipeline Design for Retrieval-Augmented Medical Question Answering
- 2024JMLR: Joint Medical LLM and Retrieval Training for Enhancing Reasoning and Professional Question Answering Capability
- 2024Development and bilingual evaluation of Japanese medical large language model within reasonably low computational resources
- 2023HyKGE: A Hypothesis Knowledge Graph Enhanced Framework for Accurate and Reliable Medical LLMs Responses