mlsanityData to 14 Oct 2026

A Survey of Spanish Clinical Language Models

arXiv 2308.02199 ↗First posted 4 Aug 2023cs.CL cs.AI cs.LGcited by 0

Not on the map. Why?

Abstract (arXiv)

This survey focuses in encoder Language Models for solving tasks in the clinical domain in the Spanish language. We review the contributions of 17 corpora focused mainly in clinical tasks, then list the most relevant Spanish Language Models and Spanish Clinical Language models. We perform a thorough comparison of these models by benchmarking them over a curated subset of the available corpora, in order to find the best-performing ones; in total more than 3000 models were fine-tuned for this study. All the tested corpora and the best models are made publically available in an accessible way, so that the results can be reproduced by independent teams or challenged in the future when new Spanish Clinical Language models are created.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 36 of its 96 references that are in the map. First posted on arXiv 4 Aug 2023.

Builds on

36 of 96 references.

References · 36

  1. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,399
  2. 2020Language Models are Few-Shot Learnerscited by 32,419
  3. 2022Training language models to follow instructions with human feedbackcited by 15,971
  4. 2023LLaMA: Open and Efficient Foundation Language Modelscited by 14,132
  5. 2019RoBERTa: A Robustly Optimized BERT Pretraining Approachcited by 13,372
  6. 2023Llama 2: Open Foundation and Fine-Tuned Chat Modelscited by 13,028
  7. 2023Judging LLM-as-a-Judge with MT-Bench and Chatbot Arenacited by 8,626
  8. 2022PaLM: Scaling Language Modeling with Pathwayscited by 5,326
  9. 2013Efficient Estimation of Word Representations in Vector Spacecited by 4,375
  10. 2014Learning Phrase Representations using RNN Encoder-Decoder for Statistical Machine Translationcited by 4,057
  11. 2019Unsupervised Cross-lingual Representation Learning at Scalecited by 3,935
  12. 2022Training Compute-Optimal Large Language Modelscited by 3,019
24 more
  1. 2018Deep contextualized word representationscited by 2,290
  2. 2019HuggingFace's Transformers: State-of-the-art Natural Language Processingcited by 2,218
  3. 2020DeBERTa: Decoding-enhanced BERT with Disentangled Attentioncited by 2,181
  4. 2019BioBERT: a pre-trained biomedical language representation model for biomedical text miningcited by 1,947
  5. 2020mT5: A massively multilingual pre-trained text-to-text transformercited by 1,818
  6. 2016Enriching Word Vectors with Subword Informationcited by 1,583
  7. 2020Don't Stop Pretraining: Adapt Language Models to Domains and Taskscited by 1,565
  8. 2021DeBERTaV3: Improving DeBERTa using ELECTRA-Style Pre-Training with Gradient-Disentangled Embedding Sharingcited by 1,254
  9. 2020Domain-Specific Language Model Pretraining for Biomedical Natural Language Processingcited by 1,009
  10. 2023A Multitask, Multilingual, Multimodal Evaluation of ChatGPT on Reasoning, Hallucination, and Interactivitycited by 894
  11. 2023Capabilities of GPT-4 on Medical Challenge Problemscited by 638
  12. 2017In-Datacenter Performance Analysis of a Tensor Processing Unitcited by 618
  13. 2022BioGPT: Generative Pre-trained Transformer for Biomedical Text Generation and Miningcited by 538
  14. 2021Datasets: A Community Library for Natural Language Processingcited by 530
  15. 2023A Comprehensive Survey on Pretrained Foundation Models: A History from BERT to ChatGPTcited by 368
  16. 2022Towards a Cleaner Document-Oriented Multilingual Crawled Corpuscited by 143
  17. 2023ClinicalGPT: Large Language Models Finetuned with Diverse Medical Data and Comprehensive Evaluationcited by 88
  18. 2023Benchmarking large language models for biomedical natural language processing applications and recommendationscited by 78
  19. 2023DrBERT: A Robust Pre-trained Model in French for Biomedical and Clinical domainscited by 27
  20. 2022Lessons learned from the evaluation of Spanish Language Modelscited by 8
  21. 2023Has It All Been Solved? Open NLP Research Questions Not Solved by Large Language Modelscited by 7
  22. 2022RigoBERTa: A State-of-the-Art Language Model For Spanishcited by 5
  23. 2020NUBES: A Corpus of Negation and Uncertainty in Spanish Clinical Textscited by 4
  24. 2021Spanish Biomedical Crawled Corpus: A Large, Diverse Dataset for Spanish Biomedical Language Modelscited by 4

Built on by

Not cited yet.