# Spanish Biomedical Crawled Corpus: A Large, Diverse Dataset for Spanish Biomedical Language Models

arXiv: https://arxiv.org/abs/2109.07765 · HTML: https://mlsanity.com/paper/2109.07765/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- Builds on 5 of its 10 references that are in the map.
- First posted on arXiv 16 Sep 2021.

## Builds on (most cited first, 5 of 5)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [BioBERT: a pre-trained biomedical language representation model for biomedical text mining](https://mlsanity.com/paper/1901.08746/) (2019)
- [SciBERT: A Pretrained Language Model for Scientific Text](https://mlsanity.com/paper/1903.10676/) (2019)
- [A Monolingual Approach to Contextualized Word Embeddings for Mid-Resource Languages](https://mlsanity.com/paper/2006.06202/) (2020)
- [Are Multilingual Models the Best Choice for Moderately Under-resourced Languages? A Comprehensive Assessment for Catalan](https://mlsanity.com/paper/2107.07903/) (2021)

## Built on by (most cited first, 4 of 4)

- [Apollo: A Lightweight Multilingual Medical LLM towards Democratizing Medical AI to 6B People](https://mlsanity.com/paper/2403.03640/) (2024)
- [Efficiently Democratizing Medical LLMs for 50 Languages via a Mixture of Language Family Experts](https://mlsanity.com/paper/2410.10626/) (2024)
- [EriBERTa: A Bilingual Pre-Trained Language Model for Clinical Natural Language Processing](https://mlsanity.com/paper/2306.07373/) (2023)
- [A Survey of Spanish Clinical Language Models](https://mlsanity.com/paper/2308.02199/) (2023)

## Abstract (arXiv)

We introduce CoWeSe (the Corpus Web Salud Espa~nol), the largest Spanish biomedical corpus to date, consisting of 4.5GB (about 750M tokens) of clean plain text. CoWeSe is the result of a massive crawler on 3000 Spanish domains executed in 2020. The corpus is openly available and already preprocessed. CoWeSe is an important resource for biomedical and health NLP in Spanish and has already been employed to train domain-specific language models and to produce word embbedings. We released the CoWeSe corpus under a Creative Commons Attribution 4.0 International license, both in Zenodo (urlhttps://zenodo.org/record/4561971#.YTI5SnVKiEA).