Teaching LLMs Brazilian Healthcare: Injecting Knowledge from Official Clinical Guidelines
Not on the map. Why?
Abstract (arXiv)
Brazil's Unified Health System (SUS) relies on official clinical guidelines that define diagnostic criteria, treatments, dosages, and monitoring procedures for over 200 million citizens. Yet current LLMs perform poorly on this guideline-specific knowledge, and no benchmark evaluates clinical recall grounded in Brazilian Portuguese protocols. We address this gap by adapting Qwen2.5-14B-Instruct to the Brazilian clinical domain. From 178 official guidelines (~5.4M tokens), we generate ~70M tokens of synthetic data in three formats -- rephrases, wiki-style articles, and question-answer pairs -- using four generator LLMs. We then apply continual pre-training followed by Group Relative Policy Optimization (GRPO). We introduce HealthBench-BR, with 1,780 balanced true/false clinical assertions, and PCDT-QA, with 890 open-ended clinical questions scored by an LLM judge. Our best model achieves 83.9% on HealthBench-BR and 85.4% on PCDT-QA, outperforming GPT-5.2, Claude Sonnet 4.6, Gemini 3.1 Pro, and Google AI Overview's web-grounded RAG despite having only 14B parameters. Ablations show that generator diversity and reinforcement learning are critical to these gains. We release all datasets, benchmarks, and model weights to support reproducible clinical NLP research for Brazilian Portuguese. Code, data, and model weights are available at https://github.com/hugoabonizio/clinical-protocols-br
Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 25 of its 35 references that are in the map. First posted on arXiv 1 May 2026.
Builds on
25 of 35 references.
References · 25
- 2021LoRA: Low-Rank Adaptation of Large Language Modelscited by 14,084
- 2024DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Modelscited by 8,851
- 2024Qwen2.5 Technical Reportcited by 4,713
- 2022Large Language Models Encode Clinical Knowledgecited by 1,807
- 2023Instruction-Following Evaluation for Large Language Modelscited by 1,143
- 2024The FineWeb Datasets: Decanting the Web for the Finest Text Data at Scalecited by 1,104
- 2023Capabilities of GPT-4 on Medical Challenge Problemscited by 630
- 2025MedGemma Technical Reportcited by 398
- 2024LoRA Learns Less and Forgets Lesscited by 313
- 2024BioMistral: A Collection of Open-Source Pretrained Large Language Models for Medical Domainscited by 299
- 2023MEDITRON-70B: Scaling Medical Pretraining for Large Language Modelscited by 272
- 2024Capabilities of Gemini Models in Medicinecited by 231
13 more
- 2024Rephrasing the Web: A Recipe for Compute and Data-Efficient Language Modelingcited by 156
- 2024Nemotron-CC: Transforming Common Crawl into a Refined Long-Horizon Pretraining Datasetcited by 153
- 2024LoRA vs Full Fine-tuning: An Illusion of Equivalencecited by 102
- 2023AlpaCare:Instruction-tuned Large Language Models for Medical Applicationcited by 88
- 2023Adapting Large Language Models to Domains via Reading Comprehensioncited by 48
- 2025Spurious Forgetting in Continual Learning of Language Modelscited by 47
- 2024Scaling Laws for Forgetting When Fine-Tuning Large Language Modelscited by 36
- 2025Scaling Laws for Forgetting during Finetuning with Pretraining Data Injectioncited by 35
- 2023BLUEX: A benchmark based on Brazilian Leading Universities Entrance eXamscited by 15
- 2025Evaluating Parameter Efficient Methods for RLVRcited by 12
- 2025HealthQA-BR: A System-Wide Benchmark Reveals Critical Knowledge Gaps in Large Language Modelscited by 3
- 2024Adapting LLMs for the Medical Domain in Portuguese: A Study on Fine-Tuning and Model Evaluationcited by 2
- 2025MedPT: A Massive Medical Question Answering Dataset for Brazilian-Portuguese Speakerscited by 2
Built on by
Not cited yet.
Similar, not cited
Close in wording, no citation either way
- 2026Beyond English benchmarks: clinical llm evaluation in Brazilian Portuguese
- 2025Reviewing Clinical Knowledge in Medical Large Language Models: Training and Beyond
- 2025Evaluating Large Language Models for Evidence-Based Clinical Question Answering
- 2026LLM-as-a-Judge in Healthcare: A Scoping Analysis of Applications, Methods, and Human Alignment ↗
- 2024Answering real-world clinical questions using large language model based systems
- 2025MIRIAD: Augmenting LLMs with millions of medical query-response pairs
- 2025MedGUIDE: Benchmarking Clinical Decision-Making in Large Language Models
- 2025Grounding Large Language Models in Clinical Evidence: A Retrieval-Augmented Generation System for Querying UK NICE Clinical Guidelines