# Neural Word Segmentation with Rich Pretraining

arXiv: https://arxiv.org/abs/1704.08960 · HTML: https://mlsanity.com/paper/1704.08960/

## Key facts

- Cited by 10 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 28 Apr 2017.

## Built on by (most cited first, 10 of 10)

- [Lexicon Enhanced Chinese Sequence Labeling Using BERT Adapter](https://mlsanity.com/paper/2105.07148/) (2021)
- [Joint Optimization of Tokenization and Downstream Model](https://mlsanity.com/paper/2105.12410/) (2021)
- [That Slepen Al the Nyght with Open Ye! Cross-era Sequence Segmentation with Switch-memory](https://mlsanity.com/paper/2209.02967/) (2022)
- [DyLex: Incorporating Dynamic Lexicons into BERT for Sequence Labeling](https://mlsanity.com/paper/2109.08818/) (2021)
- [GNN-SL: Sequence Labeling Based on Nearest Examples via GNN](https://mlsanity.com/paper/2212.02017/) (2022)
- [Neural Supervised Domain Adaptation by Augmenting Pre-trained Models with Random Units](https://mlsanity.com/paper/2106.04935/) (2021)
- [Chinese Word Segmentation with Heterogeneous Graph Neural Network](https://mlsanity.com/paper/2201.08975/) (2022)
- [Rethinking the Value of Gazetteer in Chinese Named Entity Recognition](https://mlsanity.com/paper/2207.02802/) (2022)
- [NFLAT: Non-Flat-Lattice Transformer for Chinese Named Entity Recognition](https://mlsanity.com/paper/2205.05832/) (2022)
- [Chinese Sequence Labeling with Semi-Supervised Boundary-Aware Language Model Pre-training](https://mlsanity.com/paper/2404.05560/) (2024)

## Abstract (arXiv)

Neural word segmentation research has benefited from large-scale raw texts by leveraging them for pretraining character and word embeddings. On the other hand, statistical segmentation research has exploited richer sources of external information, such as punctuation, automatic segmentation and POS. We investigate the effectiveness of a range of external training sources for neural word segmentation by building a modular segmentation model, pretraining the most important submodule using rich external sources. Results show that such pretraining significantly improves the model, leading to accuracies competitive to the best methods on six benchmarks.