# Lexicon Enhanced Chinese Sequence Labeling Using BERT Adapter

arXiv: https://arxiv.org/abs/2105.07148 · HTML: https://mlsanity.com/paper/2105.07148/

## Key facts

- Cited by 34 papers in the map (data through 14 Oct 2026).
- Builds on 24 of its 59 references that are in the map.
- First posted on arXiv 15 May 2021.

## Builds on (most cited first, 24 of 24)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Parameter-Efficient Transfer Learning for NLP](https://mlsanity.com/paper/1902.00751/) (2019)
- [Neural Architectures for Named Entity Recognition](https://mlsanity.com/paper/1603.01360/) (2016)
- [MAD-X: An Adapter-Based Framework for Multi-Task Cross-Lingual Transfer](https://mlsanity.com/paper/2005.00052/) (2020)
- [End-to-end Sequence Labeling via Bi-directional LSTM-CNNs-CRF](https://mlsanity.com/paper/1603.01354/) (2016)
- [K-Adapter: Infusing Knowledge into Pre-Trained Models with Adapters](https://mlsanity.com/paper/2002.01808/) (2020)
- [ERNIE: Enhanced Representation through Knowledge Integration](https://mlsanity.com/paper/1904.09223/) (2019)
- [Simple, Scalable Adaptation for Neural Machine Translation](https://mlsanity.com/paper/1909.08478/) (2019)
- [ERNIE 2.0: A Continual Pre-training Framework for Language Understanding](https://mlsanity.com/paper/1907.12412/) (2019)
- [Named Entity Recognition with Bidirectional LSTM-CNNs](https://mlsanity.com/paper/1511.08308/) (2015)
- [Assessing BERT's Syntactic Abilities](https://mlsanity.com/paper/1901.05287/) (2019)
- [Chinese NER Using Lattice LSTM](https://mlsanity.com/paper/1805.02023/) (2018)
- [FLAT: Chinese NER Using Flat-Lattice Transformer](https://mlsanity.com/paper/2004.11795/) (2020)
- [Glyce: Glyph-vectors for Chinese Character Representations](https://mlsanity.com/paper/1901.10125/) (2019)
- [ZEN: Pre-training Chinese Text Encoder Enhanced by N-gram Representations](https://mlsanity.com/paper/1911.00720/) (2019)
- [Simplify the Usage of Lexicon in Chinese NER](https://mlsanity.com/paper/1908.05969/) (2019)
- [Character-based Joint Segmentation and POS Tagging for Chinese using Bidirectional RNN-CRF](https://mlsanity.com/paper/1704.01314/) (2017)
- [Neural Word Segmentation with Rich Pretraining](https://mlsanity.com/paper/1704.08960/) (2017)
- [State-of-the-art Chinese Word Segmentation with Bi-LSTMs](https://mlsanity.com/paper/1808.06511/) (2018)
- [Subword Encoding in Lattice LSTM for Chinese Word Segmentation](https://mlsanity.com/paper/1810.12594/) (2018)
- [CAN-NER: Convolutional Attention Network for Chinese Named Entity Recognition](https://mlsanity.com/paper/1904.02141/) (2019)
- [Combining Discrete and Neural Features for Sequence Labeling](https://mlsanity.com/paper/1708.07279/) (2017)
- [BERT Meets Chinese Word Segmentation](https://mlsanity.com/paper/1909.09292/) (2019)
- [Investigating Self-Attention Network for Chinese Word Segmentation](https://mlsanity.com/paper/1907.11512/) (2019)

## Built on by (most cited first, 30 of 34)

- [A Benchmark for Automatic Medical Consultation System: Frameworks, Tasks and Datasets](https://mlsanity.com/paper/2204.08997/) (2022)
- [METS-CoV: A Dataset of Medical Entity and Targeted Sentiment on COVID-19 Related Tweets](https://mlsanity.com/paper/2209.13773/) (2022)
- [LERT: A Linguistically-motivated Pre-trained Language Model](https://mlsanity.com/paper/2211.05344/) (2022)
- [PUnifiedNER: A Prompting-based Unified NER System for Diverse Datasets](https://mlsanity.com/paper/2211.14838/) (2022)
- [XRAG: Cross-lingual Retrieval-Augmented Generation](https://mlsanity.com/paper/2505.10089/) (2025)
- [Delving Deep into Regularity: A Simple but Effective Method for Chinese Named Entity Recognition](https://mlsanity.com/paper/2204.05544/) (2022)
- [PaDeLLM-NER: Parallel Decoding in Large Language Models for Named Entity Recognition](https://mlsanity.com/paper/2402.04838/) (2024)
- [Medical Dialogue: A Survey of Categories, Methods, Evaluation and Challenges](https://mlsanity.com/paper/2405.10630/) (2024)
- [TURNER: The Uncertainty-based Retrieval Framework for Chinese NER](https://mlsanity.com/paper/2202.09022/) (2022)
- [TransferTOD: A Generalizable Chinese Multi-Domain Task-Oriented Dialogue System with Transfer Capabilities](https://mlsanity.com/paper/2407.21693/) (2024)
- [Neural Lexicon Reader: Reduce Pronunciation Errors in End-to-end TTS by Leveraging External Textual Knowledge](https://mlsanity.com/paper/2110.09698/) (2021)
- [Robust Self-Augmentation for Named Entity Recognition with Meta Reweighting](https://mlsanity.com/paper/2204.11406/) (2022)
- [Chinese Word Segmentation with Heterogeneous Graph Neural Network](https://mlsanity.com/paper/2201.08975/) (2022)
- [Qtrade AI at SemEval-2022 Task 11: An Unified Framework for Multilingual NER Task](https://mlsanity.com/paper/2204.07459/) (2022)
- [Structural Bias for Aspect Sentiment Triplet Extraction](https://mlsanity.com/paper/2209.00820/) (2022)
- [YATO: Yet Another deep learning based Text analysis Open toolkit](https://mlsanity.com/paper/2209.13877/) (2022)
- [Unsupervised Boundary-Aware Language Model Pretraining for Chinese Sequence Labeling](https://mlsanity.com/paper/2210.15231/) (2022)
- [Adapter-TST: A Parameter Efficient Method for Multiple-Attribute Text Style Transfer](https://mlsanity.com/paper/2305.05945/) (2023)
- [Joint Modeling of Entities and Discourse Relations for Coherence Assessment](https://mlsanity.com/paper/2509.04182/) (2025)
- [Incorporating Deep Syntactic and Semantic Knowledge for Chinese Sequence Labeling with GCN](https://mlsanity.com/paper/2306.02078/) (2023)
- [Boosting Chinese ASR Error Correction with Dynamic Error Scaling Mechanism](https://mlsanity.com/paper/2308.03423/) (2023)
- [An Effective Incorporating Heterogeneous Knowledge Curriculum Learning for Sequence Labeling](https://mlsanity.com/paper/2402.13534/) (2024)
- [Green CWS: Extreme Distillation and Efficient Decode Method Towards Industrial Application](https://mlsanity.com/paper/2111.09078/) (2021)
- [Few-shot Multi-hop Question Answering over Knowledge Base](https://mlsanity.com/paper/2112.11909/) (2021)
- [NFLAT: Non-Flat-Lattice Transformer for Chinese Named Entity Recognition](https://mlsanity.com/paper/2205.05832/) (2022)
- [Dict-TTS: Learning to Pronounce with Prior Dictionary Knowledge for Text-to-Speech](https://mlsanity.com/paper/2206.02147/) (2022)
- [Exploiting Word Semantics to Enrich Character Representations of Chinese Pre-trained Models](https://mlsanity.com/paper/2207.05928/) (2022)
- [Unified Model Learning for Various Neural Machine Translation](https://mlsanity.com/paper/2305.02777/) (2023)
- [KESDT: knowledge enhanced shallow and deep Transformer for detecting adverse drug reactions](https://mlsanity.com/paper/2308.09329/) (2023)
- [Unified Lattice Graph Fusion for Chinese Named Entity Recognition](https://mlsanity.com/paper/2312.16917/) (2023)

## Abstract (arXiv)

Lexicon information and pre-trained models, such as BERT, have been combined to explore Chinese sequence labelling tasks due to their respective strengths. However, existing methods solely fuse lexicon features via a shallow and random initialized sequence layer and do not integrate them into the bottom layers of BERT. In this paper, we propose Lexicon Enhanced BERT (LEBERT) for Chinese sequence labelling, which integrates external lexicon knowledge into BERT layers directly by a Lexicon Adapter layer. Compared with the existing methods, our model facilitates deep lexicon knowledge fusion at the lower layers of BERT. Experiments on ten Chinese datasets of three tasks including Named Entity Recognition, Word Segmentation, and Part-of-Speech tagging, show that LEBERT achieves the state-of-the-art results.