# Named Entity Recognition Under Domain Shift via Metric Learning for Life Sciences

arXiv: https://arxiv.org/abs/2401.10472 · HTML: https://mlsanity.com/paper/2401.10472/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 27 of its 55 references that are in the map.
- First posted on arXiv 19 Jan 2024.

## Builds on (most cited first, 27 of 27)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Parameter-Efficient Transfer Learning for NLP](https://mlsanity.com/paper/1902.00751/) (2019)
- [Don't Stop Pretraining: Adapt Language Models to Domains and Tasks](https://mlsanity.com/paper/2004.10964/) (2020)
- [Super-NaturalInstructions: Generalization via Declarative Instructions on 1600+ NLP Tasks](https://mlsanity.com/paper/2204.07705/) (2022)
- [Large Language Models Struggle to Learn Long-Tail Knowledge](https://mlsanity.com/paper/2211.08411/) (2022)
- [AdapterHub: A Framework for Adapting Transformers](https://mlsanity.com/paper/2007.07779/) (2020)
- [Multi-Similarity Loss with General Pair Weighting for Deep Metric Learning](https://mlsanity.com/paper/1904.06627/) (2019)
- [Semi-supervised Domain Adaptation via Minimax Entropy](https://mlsanity.com/paper/1904.06487/) (2019)
- [Transfer Learning in Biomedical Natural Language Processing: An Evaluation of BERT and ELMo on Ten Benchmarking Datasets](https://mlsanity.com/paper/1906.05474/) (2019)
- [Self-Alignment Pretraining for Biomedical Entity Representations](https://mlsanity.com/paper/2010.11784/) (2020)
- [A Systematic Study and Comprehensive Evaluation of ChatGPT on Benchmark Datasets](https://mlsanity.com/paper/2305.18486/) (2023)
- [SciFive: a text-to-text transformer model for biomedical literature](https://mlsanity.com/paper/2106.03598/) (2021)
- [Automated Concatenation of Embeddings for Structured Prediction](https://mlsanity.com/paper/2010.05006/) (2020)
- [BioRED: A Rich Biomedical Relation Extraction Dataset](https://mlsanity.com/paper/2204.04263/) (2022)
- [COVID-19 Literature Knowledge Graph Construction and Drug Repurposing Report Generation](https://mlsanity.com/paper/2007.00576/) (2020)
- [Biomedical Named Entity Recognition at Scale](https://mlsanity.com/paper/2011.06315/) (2020)
- [DrBERT: A Robust Pre-trained Model in French for Biomedical and Clinical domains](https://mlsanity.com/paper/2304.00958/) (2023)
- [Optimizing Bi-Encoder for Named Entity Recognition via Contrastive Learning](https://mlsanity.com/paper/2208.14565/) (2022)
- [Transfer Learning for Named-Entity Recognition with Neural Networks](https://mlsanity.com/paper/1705.06273/) (2017)
- [Improving Self-training for Cross-lingual Named Entity Recognition with Contrastive and Prototype Learning](https://mlsanity.com/paper/2305.13628/) (2023)
- [Hardness-guided domain adaptation to recognise biomedical named entities under low-resource scenarios](https://mlsanity.com/paper/2211.05980/) (2022)
- [Split-NER: Named Entity Recognition via Two Question-Answering-based Classifications](https://mlsanity.com/paper/2310.19942/) (2023)
- [A dynamic programming algorithm for span-based nested named-entity recognition in O(n^2)](https://mlsanity.com/paper/2210.04738/) (2022)
- [Span-based Named Entity Recognition by Generating and Compressing Information](https://mlsanity.com/paper/2302.05392/) (2023)
- [AutoTriggER: Label-Efficient and Robust Named Entity Recognition with Auxiliary Trigger Extraction](https://mlsanity.com/paper/2109.04726/) (2021)
- [CoLaDa: A Collaborative Label Denoising Framework for Cross-lingual Named Entity Recognition](https://mlsanity.com/paper/2305.14913/) (2023)
- [Chem-FINESE: Validating Fine-Grained Few-shot Entity Extraction through Text Reconstruction](https://mlsanity.com/paper/2401.10189/) (2024)

## Built on by (most cited first, 2 of 2)

- [SciArena: An Open Evaluation Platform for Non-Verifiable Scientific Literature-Grounded Tasks](https://mlsanity.com/paper/2507.01001/) (2025)
- [Gene-Metabolite Association Prediction with Interactive Knowledge Transfer Enhanced Graph for Metabolite Production](https://mlsanity.com/paper/2410.18475/) (2024)

## Abstract (arXiv)

Named entity recognition is a key component of Information Extraction (IE), particularly in scientific domains such as biomedicine and chemistry, where large language models (LLMs), e.g., ChatGPT, fall short. We investigate the applicability of transfer learning for enhancing a named entity recognition model trained in the biomedical domain (the source domain) to be used in the chemical domain (the target domain). A common practice for training such a model in a few-shot learning setting is to pretrain the model on the labeled source data, and then, to finetune it on a hand-full of labeled target examples. In our experiments, we observed that such a model is prone to mislabeling the source entities, which can often appear in the text, as the target entities. To alleviate this problem, we propose a model to transfer the knowledge from the source domain to the target domain, but, at the same time, to project the source entities and target entities into separate regions of the feature space. This diminishes the risk of mislabeling the source entities as the target entities. Our model consists of two stages: 1) entity grouping in the source domain, which incorporates knowledge from annotated events to establish relations between entities, and 2) entity discrimination in the target domain, which relies on pseudo labeling and contrastive learning to enhance discrimination between the entities in the two domains. We conduct our extensive experiments across three source and three target datasets, demonstrating that our method outperforms the baselines by up to 5% absolute value.