# Green CWS: Extreme Distillation and Efficient Decode Method Towards Industrial Application

arXiv: https://arxiv.org/abs/2111.09078 · HTML: https://mlsanity.com/paper/2111.09078/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 11 of its 20 references that are in the map.
- First posted on arXiv 17 Nov 2021.

## Builds on (most cited first, 11 of 11)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Bidirectional LSTM-CRF Models for Sequence Tagging](https://mlsanity.com/paper/1508.01991/) (2015)
- [Pre-Trained Models: Past, Present and Future](https://mlsanity.com/paper/2106.07139/) (2021)
- [Lexicon Enhanced Chinese Sequence Labeling Using BERT Adapter](https://mlsanity.com/paper/2105.07148/) (2021)
- [Toward Fast and Accurate Neural Chinese Word Segmentation with Multi-Criteria Learning](https://mlsanity.com/paper/1903.04190/) (2019)
- [Subword Encoding in Lattice LSTM for Chinese Word Segmentation](https://mlsanity.com/paper/1810.12594/) (2018)
- [RethinkCWS: Is Chinese Word Segmentation a Solved Task?](https://mlsanity.com/paper/2011.06858/) (2020)
- [Masked Conditional Random Fields for Sequence Labeling](https://mlsanity.com/paper/2103.10682/) (2021)
- [Neural Chinese Word Segmentation with Dictionary Knowledge](https://mlsanity.com/paper/1807.05849/) (2018)
- [Transfer Deep Learning for Low-Resource Chinese Word Segmentation with a Novel Neural Network](https://mlsanity.com/paper/1702.04488/) (2017)
- [Attention Is All You Need for Chinese Word Segmentation](https://mlsanity.com/paper/1910.14537/) (2019)

## Abstract (arXiv)

Benefiting from the strong ability of the pre-trained model, the research on Chinese Word Segmentation (CWS) has made great progress in recent years. However, due to massive computation, large and complex models are incapable of empowering their ability for industrial use. On the other hand, for low-resource scenarios, the prevalent decode method, such as Conditional Random Field (CRF), fails to exploit the full information of the training data. This work proposes a fast and accurate CWS framework that incorporates a light-weighted model and an upgraded decode method (PCRF) towards industrially low-resource CWS scenarios. First, we distill a Transformer-based student model as an encoder, which not only accelerates the inference speed but also combines open knowledge and domain-specific knowledge. Second, the perplexity score to evaluate the language model is fused into the CRF module to better identify the word boundaries. Experiments show that our work obtains relatively high performance on multiple datasets with as low as 14% of time consumption compared with the original BERT-based model. Moreover, under the low-resource setting, we get superior results in comparison with the traditional decoding methods.