# Conceptualized Representation Learning for Chinese Biomedical Text Mining

arXiv: https://arxiv.org/abs/2008.10813 · HTML: https://mlsanity.com/paper/2008.10813/

## Key facts

- Cited by 19 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 25 Aug 2020.

## Built on by (most cited first, 19 of 19)

- [A Survey of Large Language Models for Healthcare: from Data, Technology, and Applications to Accountability and Ethics](https://mlsanity.com/paper/2310.05694/) (2023)
- [AMMUS : A Survey of Transformer-based Pretrained Models in Natural Language Processing](https://mlsanity.com/paper/2108.05542/) (2021)
- [A Comprehensive Survey of Scientific Large Language Models and Their Applications in Scientific Discovery](https://mlsanity.com/paper/2406.10833/) (2024)
- [Pre-trained Language Models in Biomedical Domain: A Systematic Survey](https://mlsanity.com/paper/2110.05006/) (2021)
- [Huatuo-26M, a Large-scale Chinese Medical QA Dataset](https://mlsanity.com/paper/2305.01526/) (2023)
- [AMMU : A Survey of Transformer-based Biomedical Pretrained Language Models](https://mlsanity.com/paper/2105.00827/) (2021)
- [Multi-CPR: A Multi Domain Chinese Dataset for Passage Retrieval](https://mlsanity.com/paper/2203.03367/) (2022)
- [SMedBERT: A Knowledge-Enhanced Pre-trained Language Model with Structured Semantics for Medical Text Mining](https://mlsanity.com/paper/2108.08983/) (2021)
- [A Survey of Pre-trained Language Models for Processing Scientific Text](https://mlsanity.com/paper/2401.17824/) (2024)
- [Building Chinese Biomedical Language Models via Multi-Level Text Discrimination](https://mlsanity.com/paper/2110.07244/) (2021)
- [Do not Mask Randomly: Effective Domain-adaptive Pre-training by Masking In-domain Keywords](https://mlsanity.com/paper/2307.07160/) (2023)
- [Normal vs. Adversarial: Salience-based Analysis of Adversarial Samples for Relation Extraction](https://mlsanity.com/paper/2104.00312/) (2021)
- [ZJUKLAB at SemEval-2021 Task 4: Negative Augmentation with Language Model for Reading Comprehension of Abstract Meaning](https://mlsanity.com/paper/2102.12828/) (2021)
- [Matching-based Term Semantics Pre-training for Spoken Patient Query Understanding](https://mlsanity.com/paper/2303.01341/) (2023)
- [Doctor Recommendation in Online Health Forums via Expertise Learning](https://mlsanity.com/paper/2203.02932/) (2022)
- [Manifold-based Verbalizer Space Re-embedding for Tuning-free Prompt-based Classification](https://mlsanity.com/paper/2309.04174/) (2023)
- [KBioXLM: A Knowledge-anchored Biomedical Multilingual Pretrained Language Model](https://mlsanity.com/paper/2311.11564/) (2023)
- [Knowledge-injected Prompt Learning for Chinese Biomedical Entity Normalization](https://mlsanity.com/paper/2308.12025/) (2023)
- [Enriched text-guided variational multimodal knowledge distillation network (VMD) for automated diagnosis of plaque vulnerability in 3D carotid artery MRI](https://mlsanity.com/paper/2509.11924/) (2025)

## Abstract (arXiv)

Biomedical text mining is becoming increasingly important as the number of biomedical documents and web data rapidly grows. Recently, word representation models such as BERT has gained popularity among researchers. However, it is difficult to estimate their performance on datasets containing biomedical texts as the word distributions of general and biomedical corpora are quite different. Moreover, the medical domain has long-tail concepts and terminologies that are difficult to be learned via language models. For the Chinese biomedical text, it is more difficult due to its complex structure and the variety of phrase combinations. In this paper, we investigate how the recently introduced pre-trained language model BERT can be adapted for Chinese biomedical corpora and propose a novel conceptualized representation learning approach. We also release a new Chinese Biomedical Language Understanding Evaluation benchmark (ChineseBLUE). We examine the effectiveness of Chinese pre-trained models: BERT, BERT-wwm, RoBERTa, and our approach. Experimental results on the benchmark show that our approach could bring significant gain. We release the pre-trained model on GitHub: https://github.com/alibaba-research/ChineseBLUE.