mlsanityData to 14 Oct 2026

Development and bilingual evaluation of Japanese medical large language model within reasonably low computational resources

arXiv 2409.11783 ↗First posted 18 Sep 2024cs.CLcited by 3

On the map: LLM reasoning → Clinical and biomedical applications

Abstract (arXiv)

The recent success of large language models (LLMs) and the scaling law has led to a widespread adoption of larger models. Particularly in the healthcare industry, there is an increasing demand for locally operated LLMs due to security concerns. However, the majority of high quality open-source LLMs have a size of 70B parameters, imposing significant financial burdens on users for GPU preparation and operation. To overcome these issues, we present a medical adaptation based on the recent 7B models, which enables the operation in low computational resources. We compare the performance on medical question-answering benchmarks in two languages (Japanese and English), demonstrating that its scores reach parity with or surpass those of currently existing medical LLMs that are ten times larger. We find that fine-tuning an English-centric base model on Japanese medical dataset improves the score in both language, supporting the effect of cross-lingual knowledge transfer. We hope that this study will alleviate financial challenges, serving as a stepping stone for clinical institutions to practically utilize LLMs locally. Our evaluation code is available at https://github.com/stardust-coder/japanese-lm-med-harness.

Key facts. Cited by 3 papers in the map (data through 14 Oct 2026). Placed in Clinical and biomedical applications, one of 17 directions in LLM reasoning. Builds on 26 of its 58 references that are in the map. First posted on arXiv 18 Sep 2024.

Builds on

26 of 58 references.

References · 26

  1. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,399
  2. 2023LLaMA: Open and Efficient Foundation Language Modelscited by 14,132
  3. 2023Llama 2: Open Foundation and Fine-Tuned Chat Modelscited by 13,028
  4. 2020Measuring Massive Multitask Language Understandingcited by 7,963
  5. 2020Scaling Laws for Neural Language Modelscited by 6,586
  6. 2023Mistral 7Bcited by 3,164
  7. 2019HuggingFace's Transformers: State-of-the-art Natural Language Processingcited by 2,218
  8. 2024Mixtral of Expertscited by 1,823
  9. 2020What Disease does this Patient Have? A Large-scale Open Domain Question Answering Dataset from Medical Examscited by 1,356
  10. 2023Capabilities of GPT-4 on Medical Challenge Problemscited by 638
  11. 2024Mixture-of-Agents Enhances Large Language Model Capabilitiescited by 438
  12. 2023Towards Expert-Level Medical Question Answering with Large Language Modelscited by 384
14 more
  1. 2024Benchmarking Retrieval-Augmented Generation for Medicinecited by 331
  2. 2023Can Generalist Foundation Models Outcompete Special-Purpose Tuning? Case Study in Medicinecited by 329
  3. 2024BioMistral: A Collection of Open-Source Pretrained Large Language Models for Medical Domainscited by 301
  4. 2024Capabilities of Gemini Models in Medicinecited by 237
  5. 2024Evolutionary Optimization of Model Merging Recipescited by 208
  6. 2020Byte Pair Encoding is Suboptimal for Language Model Pretrainingcited by 192
  7. 2023CMB: A Comprehensive Medical Benchmark in Chinesecited by 115
  8. 2024Continual Pre-Training for Cross-Lingual LLM Adaptation: Enhancing Japanese Language Capabilitiescited by 111
  9. 2024Towards Building Multilingual Language Model for Medicinecited by 92
  10. 2023Evaluating GPT-4 and ChatGPT on Japanese Medical Licensing Examinationscited by 63
  11. 2024Should We Respect LLMs? A Cross-Lingual Study on the Influence of Prompt Politeness on LLM Performancecited by 41
  12. 2024MedFuzz: Exploring the Robustness of Large Language Models in Medical Question Answeringcited by 26
  13. 2024BiMediX: Bilingual Medical Mixture of Experts LLMcited by 21
  14. 202470B-parameter large language models in Japanese medical question-answeringcited by 3

Built on by

Cited by year

Citing papers per year: 2025: 3.

3 papers.

  1. 2025Stabilizing Reasoning in Medical LLMs with Continued Pretraining and Reasoning Preference Optimizationcited by 4
  2. 2025A Japanese Language Model and Three New Evaluation Benchmarks for Pharmaceutical NLPcited by 2
  3. 2025MedRECT: A Medical Reasoning Benchmark for Error Correction in Clinical Textscited by 2

Similar, not cited

Close in wording, no citation either way

  1. 2026Assessing Large Language Models for Medical QA: Zero-Shot and LLM-as-a-Judge Evaluation
  2. 2025PersianMedQA: Evaluating Large Language Models on a Persian-English Bilingual Medical Question Answering Benchmark
  3. 2026To Adapt or not to Adapt, Rethinking the Value of Medical Knowledge-Aware Large Language Models
  4. 2024MedExpQA: Multilingual Benchmarking of Large Language Models for Medical Question Answering
  5. 2025Exploring the Role of Knowledge Graph-Based RAG in Japanese Medical Question Answering with Small-Scale LLMs
  6. 2024A Survey for Large Language Models in Biomedicine
  7. 2023Benchmarking Large Language Models on CMExam -- A Comprehensive Chinese Medical Exam Dataset
  8. 2025MKG-Rank: Enhancing Large Language Models with Knowledge Graph for Multilingual Medical Question Answering