# Efficient Fine-Tuning of Large Language Models for Automated Medical Documentation

arXiv: https://arxiv.org/abs/2409.09324 · HTML: https://mlsanity.com/paper/2409.09324/

## Key facts

- Cited by 16 papers in the map (data through 14 Oct 2026).
- Builds on 12 of its 57 references that are in the map.
- First posted on arXiv 14 Sep 2024.

## Builds on (most cited first, 12 of 12)

- [LLaMA: Open and Efficient Foundation Language Models](https://mlsanity.com/paper/2302.13971/) (2023)
- [QLoRA: Efficient Finetuning of Quantized LLMs](https://mlsanity.com/paper/2305.14314/) (2023)
- [Mistral 7B](https://mlsanity.com/paper/2310.06825/) (2023)
- [Gemma: Open Models Based on Gemini Research and Technology](https://mlsanity.com/paper/2403.08295/) (2024)
- [ODTrack: Online Dense Temporal Token Learning for Visual Tracking](https://mlsanity.com/paper/2401.01686/) (2024)
- [Large language models in healthcare and medical domain: A review](https://mlsanity.com/paper/2401.06775/) (2023)
- [Towards Unified Token Learning for Vision-Language Tracking](https://mlsanity.com/paper/2308.14103/) (2023)
- [Understanding the Dynamics of DNNs Using Graph Modularity](https://mlsanity.com/paper/2111.12485/) (2021)
- [CMAT: A Multi-Agent Collaboration Tuning Framework for Enhancing Small Language Models](https://mlsanity.com/paper/2404.01663/) (2024)
- [Robust Domain Generalization for Multi-modal Object Recognition](https://mlsanity.com/paper/2408.05831/) (2024)
- [MathNAS: If Blocks Have a Role in Mathematical Architecture Design](https://mlsanity.com/paper/2311.04943/) (2023)
- [Enhancing Summarization Performance through Transformer-Based Prompt Engineering in Automated Medical Reporting](https://mlsanity.com/paper/2311.13274/) (2023)

## Built on by (most cited first, 16 of 16)

- [The Rise of Small Language Models in Healthcare: A Comprehensive Survey](https://mlsanity.com/paper/2504.17119/) (2025)
- [Deep Reinforcement Learning-based Obstacle Avoidance for Robot Movement in Warehouse Environments](https://mlsanity.com/paper/2409.14972/) (2024)
- [A GEN AI Framework for Medical Note Generation](https://mlsanity.com/paper/2410.01841/) (2024)
- [CATCH: A Modular Cross-domain Adaptive Template with Hook](https://mlsanity.com/paper/2510.26582/) (2025)
- [Optimization and Application of Cloud-based Deep Learning Architecture for Multi-Source Data Prediction](https://mlsanity.com/paper/2410.12642/) (2024)
- [Beyond Multiple-Choice Accuracy: Real-World Challenges of Implementing Large Language Models in Healthcare](https://mlsanity.com/paper/2410.18460/) (2024)
- [Efficient Cold-Start Recommendation via BPE Token-Level Embedding Initialization with LLM](https://mlsanity.com/paper/2509.13179/) (2025)
- [The Structural Scalpel: Automated Contiguous Layer Pruning for Large Language Models](https://mlsanity.com/paper/2510.23652/) (2025)
- [OC-Distill: Ontology-aware Contrastive Learning with Cross-Modal Distillation for ICU Risk Prediction](https://mlsanity.com/paper/2604.16878/) (2026)
- [Mitigating Knowledge Conflicts in Language Model-Driven Question Answering](https://mlsanity.com/paper/2411.11344/) (2024)
- [Large Language Models for Cancer Communication: Evaluating Linguistic Quality, Safety, and Accessibility in Generative AI](https://mlsanity.com/paper/2505.10472/) (2025)
- [Coarse-to-Fine Personalized LLM Impressions for Streamlined Radiology Reports](https://mlsanity.com/paper/2508.15845/) (2025)
- [Agentic AI in Healthcare & Medicine: A Seven-Dimensional Taxonomy for Empirical Evaluation of LLM-based Agents](https://mlsanity.com/paper/2602.04813/) (2026)
- [Towards Scalable SOAP Note Generation: A Weakly Supervised Multimodal Framework](https://mlsanity.com/paper/2506.10328/) (2025)
- [Skin-SOAP: A Weakly Supervised Framework for Generating Structured SOAP Notes](https://mlsanity.com/paper/2508.05019/) (2025)
- [Robust RGB-T Tracking via Learnable Visual Fourier Prompt Fine-tuning and Modality Fusion Prompt Generation](https://mlsanity.com/paper/2509.19733/) (2025)

## Abstract (arXiv)

Scientific research indicates that for every hour spent in direct patient care, physicians spend nearly two additional hours on administrative tasks, particularly on electronic health records (EHRs) and desk work. This excessive administrative burden not only reduces the time available for patient care but also contributes to physician burnout and inefficiencies in healthcare delivery. To address these challenges, this study introduces MediGen, a fine-tuned large language model (LLM) designed to automate the generation of medical reports from medical dialogues. By leveraging state-of-the-art methodologies for fine-tuning open-source pretrained models, including LLaMA3-8B, MediGen achieves high accuracy in transcribing and summarizing clinical interactions. The fine-tuned LLaMA3-8B model demonstrated promising results, achieving a ROUGE score of 58% and a BERTScore-F1 of 72%, indicating its effectiveness in generating accurate and clinically relevant medical reports. These findings suggest that MediGen has the potential to significantly reduce the administrative workload on physicians, improving both healthcare efficiency and physician well-being.