mlsanityData to 7 Oct 2026

A Continued Pretrained LLM Approach for Automatic Medical Note Generation

arXiv 2403.09057 ↗First posted 14 Mar 2024cs.CL cs.AIcited by 30

Not on the map. Why?

Abstract (arXiv)

LLMs are revolutionizing NLP tasks. However, the use of the most advanced LLMs, such as GPT-4, is often prohibitively expensive for most specialized fields. We introduce HEAL, the first continuously trained 13B LLaMA2-based LLM that is purpose-built for medical conversations and measured on automated scribing. Our results demonstrate that HEAL outperforms GPT-4 and PMC-LLaMA in PubMedQA, with an accuracy of 78.4%. It also achieves parity with GPT-4 in generating medical notes. Remarkably, HEAL surpasses GPT-4 and Med-PaLM 2 in identifying more correct medical concepts and exceeds the performance of human scribes and other comparable models in correctness and completeness.

Key facts. Cited by 30 papers in the map (data through 7 Oct 2026). Builds on 18 of its 26 references that are in the map. First posted on arXiv 14 Mar 2024.

Builds on

18 of 26 references.

References · 18

  1. 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,491
  2. 2023LLaMA: Open and Efficient Foundation Language Modelscited by 13,958
  3. 2020The Pile: An 800GB Dataset of Diverse Text for Language Modelingcited by 2,346
  4. 2022Large Language Models Encode Clinical Knowledgecited by 1,807
  5. 2020Don't Stop Pretraining: Adapt Language Models to Domains and Taskscited by 1,548
  6. 2019SciBERT: A Pretrained Language Model for Scientific Textcited by 1,404
  7. 2020What Disease does this Patient Have? A Large-scale Open Domain Question Answering Dataset from Medical Examscited by 1,324
  8. 2019PubMedQA: A Dataset for Biomedical Research Question Answeringcited by 1,271
  9. 2023Extending Context Window of Large Language Models via Positional Interpolationcited by 721
  10. 2021Deduplicating Training Data Makes Language Models Bettercited by 705
  11. 2023Capabilities of GPT-4 on Medical Challenge Problemscited by 630
  12. 2022BioGPT: Generative Pre-trained Transformer for Biomedical Text Generation and Miningcited by 531
6 more
  1. 2023Orca: Progressive Learning from Complex Explanation Traces of GPT-4cited by 369
  2. 2020MedDialog: Two Large-scale Medical Dialogue Datasetscited by 157
  3. 2023PMC-LLaMA: Towards Building Open-source Language Models for Medicinecited by 87
  4. 2021Leveraging Pretrained Models for Automatic Summarization of Doctor-Patient Conversationscited by 42
  5. 2021MedGPT: Medical Concept Prediction from Clinical Narrativescited by 24
  6. 2022Continued Pretraining for Better Zero- and Few-Shot Promptabilitycited by 10

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 10; 2025: 18; 2026: 2.

30 papers.

Sort:

By area:

  1. 2025Agentic Large Language Models, a surveycited by 105
  2. 2024Synthetic continued pretrainingcited by 56
  3. 2025Llama-3.1-FoundationAI-SecurityLLM-Base-8B Technical Reportcited by 28
  4. 2024TWIN-GPT: Digital Twins for Clinical Trials via Large Language Modelcited by 24
  5. 2025Towards User-level Private Reinforcement Learning with Human Feedbackcited by 13
  6. 2025A Case Study Exploring the Current Landscape of Synthetic Medical Record Generation with Commercial LLMscited by 13
  7. 2025Domain Adaptation of Foundation LLMs for e-Commercecited by 8
  8. 2024RoQLlama: A Lightweight Romanian Adapted Language Modelcited by 7
  9. 2025MedTVT-R1: A Multimodal LLM Empowering Medical Reasoning and Diagnosiscited by 7
  10. 2025RedOne: Revealing Domain-specific LLM Post-Training in Social Networking Servicescited by 6
  11. 2024Domain-Specific Pretraining of Language Models: A Comparative Study in the Medical Fieldcited by 5
  12. 2025Knowledge Hierarchy Guided Biological-Medical Dataset Distillation for Domain LLM Trainingcited by 5
18 more
  1. 2025EHRStruct: A Comprehensive Benchmark Framework for Evaluating Large Language Models on Structured Electronic Health Record Taskscited by 5
  2. 2025Enhancing Domain-Specific Encoder Models with LLM-Generated Data: How to Leverage Ontologies, and How to Do Without Themcited by 4
  3. 2024OrthoDoc: Multimodal Large Language Model for Assisting Diagnosis in Computed Tomographycited by 3
  4. 2025TN-Eval: Rubric and Evaluation Protocols for Measuring the Quality of Behavioral Therapy Notescited by 3
  5. 2024Watermarking Large Language Models and the Generated Content: Opportunities and Challengescited by 2
  6. 2024GeneSUM: Large Language Model-based Gene Summary Extractioncited by 2
  7. 2024Exploring LLM-based Data Annotation Strategies for Medical Dialogue Preference Alignmentcited by 1
  8. 2024ALKAFI-LLAMA3: Fine-Tuning LLMs for Precise Legal Understanding in Palestinecited by 1
  9. 2025Enhancing Hepatopathy Clinical Trial Efficiency: A Secure, Large Language Model-Powered Pre-Screening Pipelinecited by 1
  10. 2025Knowledge-Driven Agentic Scientific Corpus Distillation Framework for Biomedical Large Language Models Trainingcited by 1
  11. 2025Enhancing Clinical Note Generation with ICD-10, Clinical Ontology Knowledge Graphs, and Chain-of-Thought Prompting Using GPT-4cited by 1
  12. 2026WaterMoE: Expert-Routing-based Watermarking for High Fidelity and Efficiencycited by 1
  13. 2024Searching for Best Practices in Medical Transcription with Large Language Modelcited by 0
  14. 2025Medalyze: Lightweight Medical Report Summarization Application Using FLAN-T5-Largecited by 0
  15. 2025ChiMed 2.0: Advancing Chinese Medical Dataset in Facilitating Large Language Modelingcited by 0
  16. 2025FedSODA: Federated Fine-tuning of LLMs via Similarity Group Pruning and Orchestrated Distillation Alignmentcited by 0
  17. 2025MM-Telco: Benchmarks and Multimodal Large Language Models for Telecom Applicationscited by 0
  18. 2026Continually self-improving AIcited by 0