mlsanityData to 7 Oct 2026

Quantized Large Language Models in Biomedical Natural Language Processing: Evaluation and Recommendation

arXiv 2509.04534 ↗First posted 4 Sep 2025cs.CL cs.AIcited by 3

Not on the map. Why?

Abstract (arXiv)

Large language models have demonstrated remarkable capabilities in biomedical natural language processing, yet their rapid growth in size and computational requirements present a major barrier to adoption in healthcare settings where data privacy precludes cloud deployment and resources are limited. In this study, we systematically evaluated the impact of quantization on 12 state-of-the-art large language models, including both general-purpose and biomedical-specific models, across eight benchmark datasets covering four key tasks: named entity recognition, relation extraction, multi-label classification, and question answering. We show that quantization substantially reduces GPU memory requirements-by up to 75%-while preserving model performance across diverse tasks, enabling the deployment of 70B-parameter models on 40GB consumer-grade GPUs. In addition, domain-specific knowledge and responsiveness to advanced prompting methods are largely maintained. These findings provide significant practical and guiding value, highlighting quantization as a practical and effective strategy for enabling the secure, local deployment of large yet high-capacity language models in biomedical contexts, bridging the gap between technical advances in AI and real-world clinical translation.

Key facts. Cited by 3 papers in the map (data through 7 Oct 2026). Builds on 40 of its 51 references that are in the map. First posted on arXiv 4 Sep 2025.

Builds on

40 of 51 references.

References · 40

  1. 2020Language Models are Few-Shot Learnerscited by 32,199
  2. 2023GPT-4 Technical Reportcited by 18,132
  3. 2024The Llama 3 Herd of Modelscited by 14,192
  4. 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,302
  5. 2020Retrieval-Augmented Generation for Knowledge-Intensive NLP Taskscited by 9,160
  6. 2020Scaling Laws for Neural Language Modelscited by 6,411
  7. 2022Self-Consistency Improves Chain of Thought Reasoning in Language Modelscited by 5,601
  8. 2021Finetuned Language Models Are Zero-Shot Learnerscited by 3,747
  9. 2019HuggingFace's Transformers: State-of-the-art Natural Language Processingcited by 2,201
  10. 2020What Disease does this Patient Have? A Large-scale Open Domain Question Answering Dataset from Medical Examscited by 1,324
  11. 2019PubMedQA: A Dataset for Biomedical Research Question Answeringcited by 1,271
  12. 2022LLM.int8(): 8-bit Matrix Multiplication for Transformers at Scalecited by 881
28 more
  1. 2021A Survey of Quantization Methods for Efficient Neural Network Inferencecited by 798
  2. 2024Phi-4 Technical Reportcited by 675
  3. 2024DeepSeek LLM: Scaling Open-Source Language Models with Longtermismcited by 621
  4. 20218-bit Optimizers via Block-wise Quantizationcited by 395
  5. 2023MEDITRON-70B: Scaling Medical Pretraining for Large Language Modelscited by 272
  6. 2024HuatuoGPT-o1, Towards Medical Complex Reasoning with LLMscited by 257
  7. 2023QA-LoRA: Quantization-Aware Low-Rank Adaptation of Large Language Modelscited by 154
  8. 2024Med42-v2: A Suite of Clinical LLMscited by 89
  9. 2023Benchmarking large language models for biomedical natural language processing applications and recommendationscited by 78
  10. 2024Evaluating Quantized Large Language Modelscited by 75
  11. 2023Clinical Camel: An Open Expert-Level Medical Language Model with Dialogue-Based Knowledge Encodingcited by 71
  12. 2024Large Language Models for Disease Diagnosis: A Scoping Reviewcited by 51
  13. 2024A Comprehensive Evaluation of Quantization Strategies for Large Language Modelscited by 47
  14. 2024How Does Quantization Affect Multilingual LLMs?cited by 41
  15. 2025Quantization Meets Reasoning: Exploring LLM Low-Bit Quantization Degradation for Mathematical Reasoningcited by 36
  16. 2025Sustainable LLM Inference for Edge AI: Evaluating Quantized LLMs for Energy Efficiency, Output Accuracy, and Inference Latencycited by 32
  17. 2025Automating Expert-Level Medical Reasoning Evaluation of Large Language Modelscited by 21
  18. 2022Multi-label classification for biomedical literature: an overview of the BioCreative VII LitCovid Track for COVID-19 literature topic annotationscited by 14
  19. 2024Prompt engineering paradigms for medical applications: scoping review and recommendations for better practicescited by 13
  20. 2025MMRAG: Multi-Mode Retrieval-Augmented Generation with Large Language Models for Biomedical In-Context Learningcited by 13
  21. 2024Evaluating the Generalization Ability of Quantized LLMs: Benchmark, Analysis, and Toolboxcited by 11
  22. 2024RAMIE: Retrieval-Augmented Multi-task Information Extraction with Large Language Models on Dietary Supplementscited by 11
  23. 2025Optimizing LLMs for Resource-Constrained Environments: A Survey of Model Compression Techniquescited by 9
  24. 2025Uncertainty-Aware Large Language Models for Explainable Disease Diagnosiscited by 9
  25. 2024Towards Better Multi-task Learning: A Framework for Optimizing Dataset Combinations in Large Language Modelscited by 6
  26. 2025An evaluation of DeepSeek Models in Biomedical Natural Language Processingcited by 6
  27. 2025Retrieval-augmented in-context learning for multimodal large language models in disease classificationcited by 4
  28. 2025Data-Efficient Biomedical In-Context Learning: A Diversity-Enhanced Submodular Perspectivecited by 1

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2026: 3.

3 papers.

  1. 2026The Illusion of Equivalency: Statistical Characterization of Quantization Effects in LLMscited by 3
  2. 2026EpiScreen: Early Epilepsy Detection from Electronic Health Records with Large Language Modelscited by 0
  3. 2026The Effect of Quantization on Clinical Benchmarks: Accuracy and Safety Across Model Familiescited by 0