# HunFlair2 in a cross-corpus evaluation of biomedical named entity recognition and normalization tools arXiv: https://arxiv.org/abs/2402.12372 ยท HTML: https://mlsanity.com/paper/2402.12372/ ## Key facts - Cited by 8 papers in the map (data through 14 Oct 2026). - Builds on 36 of its 134 references that are in the map. - First posted on arXiv 19 Feb 2024. ## Builds on (most cited first, 30 of 36) - [Training language models to follow instructions with human feedback](https://mlsanity.com/paper/2203.02155/) (2022) - [RoBERTa: A Robustly Optimized BERT Pretraining Approach](https://mlsanity.com/paper/1907.11692/) (2019) - [BioBERT: a pre-trained biomedical language representation model for biomedical text mining](https://mlsanity.com/paper/1901.08746/) (2019) - [Domain-Specific Language Model Pretraining for Biomedical Natural Language Processing](https://mlsanity.com/paper/2007.15779/) (2020) - [Neural Architectures for Named Entity Recognition](https://mlsanity.com/paper/1603.01360/) (2016) - [Transfer Learning in Biomedical Natural Language Processing: An Evaluation of BERT and ELMo on Ten Benchmarking Datasets](https://mlsanity.com/paper/1906.05474/) (2019) - [ScispaCy: Fast and Robust Models for Biomedical Natural Language Processing](https://mlsanity.com/paper/1902.07669/) (2019) - [Self-Alignment Pretraining for Biomedical Entity Representations](https://mlsanity.com/paper/2010.11784/) (2020) - [LinkBERT: Pretraining Language Models with Document Links](https://mlsanity.com/paper/2203.15827/) (2022) - [Pre-trained Language Models in Biomedical Domain: A Systematic Survey](https://mlsanity.com/paper/2110.05006/) (2021) - [BioRED: A Rich Biomedical Relation Extraction Dataset](https://mlsanity.com/paper/2204.04263/) (2022) - [MedMentions: A Large Biomedical Corpus Annotated with UMLS Concepts](https://mlsanity.com/paper/1902.09476/) (2019) - [Multi-domain Clinical Natural Language Processing with MedCAT: the Medical Concept Annotation Toolkit](https://mlsanity.com/paper/2010.01165/) (2020) - [Biomedical Entity Representations with Synonym Marginalization](https://mlsanity.com/paper/2005.00239/) (2020) - [Evaluation Measures for Hierarchical Classification: a unified view and novel approaches](https://mlsanity.com/paper/1306.6802/) (2013) - [BERN2: an advanced neural biomedical named entity recognition and normalization tool](https://mlsanity.com/paper/2201.02080/) (2022) - [Launching into clinical space with medspaCy: a new clinical text processing toolkit in Python](https://mlsanity.com/paper/2106.07799/) (2021) - [T-NER: An All-Round Python Library for Transformer-based Named Entity Recognition](https://mlsanity.com/paper/2209.12616/) (2022) - [BigBIO: A Framework for Data-Centric Biomedical Natural Language Processing](https://mlsanity.com/paper/2206.15076/) (2022) - [Generative Biomedical Entity Linking via Knowledge Base-Guided Pre-training and Synonyms-Aware Fine-tuning](https://mlsanity.com/paper/2204.05164/) (2022) - [Knowledge-Rich Self-Supervision for Biomedical Entity Linking](https://mlsanity.com/paper/2112.07887/) (2021) - [Biomedical Named Entity Recognition at Scale](https://mlsanity.com/paper/2011.06315/) (2020) - [HunFlair: An Easy-to-Use Tool for State-of-the-Art Biomedical Named Entity Recognition](https://mlsanity.com/paper/2008.07347/) (2020) - [NeuroNER: an easy-to-use program for named-entity recognition based on neural networks](https://mlsanity.com/paper/1705.05487/) (2017) - [Entity Linking and Discovery via Arborescence-based Supervised Clustering](https://mlsanity.com/paper/2109.01242/) (2021) - [Fast and Effective Biomedical Entity Linking Using a Dual Encoder](https://mlsanity.com/paper/2103.05028/) (2021) - [Med7: a transferable clinical natural language processing model for electronic health records](https://mlsanity.com/paper/2003.01271/) (2020) - [Cross-Domain Data Integration for Named Entity Disambiguation in Biomedical Text](https://mlsanity.com/paper/2110.08228/) (2021) - [AIONER: All-in-one scheme-based biomedical named entity recognition using deep learning](https://mlsanity.com/paper/2211.16944/) (2022) - [A Lightweight Neural Model for Biomedical Entity Linking](https://mlsanity.com/paper/2012.08844/) (2020) ## Built on by (most cited first, 8 of 8) - [Large Language Model Benchmarks in Medical Tasks](https://mlsanity.com/paper/2410.21348/) (2024) - [Patience is all you need! An agentic system for performing scientific literature review](https://mlsanity.com/paper/2504.08752/) (2025) - [Privacy Meets Explainability: Managing Confidential Data and Transparency Policies in LLM-Empowered Science](https://mlsanity.com/paper/2504.09961/) (2025) - [Towards Autonomous Mechanistic Reasoning in Virtual Cells](https://mlsanity.com/paper/2604.11661/) (2026) - [Effective Multi-Task Learning for Biomedical Named Entity Recognition](https://mlsanity.com/paper/2507.18542/) (2025) - [Asking the Right Questions: Ontology-Grounded Interpretable Embeddings for Biomedical Text](https://mlsanity.com/paper/2603.01690/) (2026) - [Few-Shot Biomedical Relation Extraction with Large Language Models: A Viable Alternative to Supervised Learning?](https://mlsanity.com/paper/2606.15412/) (2026) - [BELXTR: Biomedical Entity Linking via Contextualized Token Retrieval](https://mlsanity.com/paper/2609.25859/) (2026) ## Abstract (arXiv) With the exponential growth of the life science literature, biomedical text mining (BTM) has become an essential technology for accelerating the extraction of insights from publications. Identifying named entities (e.g., diseases, drugs, or genes) in texts and their linkage to reference knowledge bases are crucial steps in BTM pipelines to enable information aggregation from different documents. However, tools for these two steps are rarely applied in the same context in which they were developed. Instead, they are applied in the wild, i.e., on application-dependent text collections different from those used for the tools' training, varying, e.g., in focus, genre, style, and text type. This raises the question of whether the reported performance of BTM tools can be trusted for downstream applications. Here, we report on the results of a carefully designed cross-corpus benchmark for named entity extraction, where tools were applied systematically to corpora not used during their training. Based on a survey of 28 published systems, we selected five for an in-depth analysis on three publicly available corpora encompassing four different entity types. Comparison between tools results in a mixed picture and shows that, in a cross-corpus setting, the performance is significantly lower than the one reported in an in-corpus setting. HunFlair2 showed the best performance on average, being closely followed by PubTator. Our results indicate that users of BTM tools should expect diminishing performances when applying them in the wild compared to original publications and show that further research is necessary to make BTM tools more robust.