# RethinkCWS: Is Chinese Word Segmentation a Solved Task?

arXiv: https://arxiv.org/abs/2011.06858 · HTML: https://mlsanity.com/paper/2011.06858/

## Key facts

- Cited by 7 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 13 Nov 2020.

## Built on by (most cited first, 7 of 7)

- [MuCGEC: a Multi-Reference Multi-Source Evaluation Dataset for Chinese Grammatical Error Correction](https://mlsanity.com/paper/2204.10994/) (2022)
- [Towards More Fine-grained and Reliable NLP Performance Prediction](https://mlsanity.com/paper/2102.05486/) (2021)
- [Larger-Context Tagging: When and Why Does It Work?](https://mlsanity.com/paper/2104.04434/) (2021)
- [Zero-shot hashtag segmentation for multilingual sentiment analysis](https://mlsanity.com/paper/2112.03213/) (2021)
- [Incorporating Deep Syntactic and Semantic Knowledge for Chinese Sequence Labeling with GCN](https://mlsanity.com/paper/2306.02078/) (2023)
- [Green CWS: Extreme Distillation and Efficient Decode Method Towards Industrial Application](https://mlsanity.com/paper/2111.09078/) (2021)
- [An Evaluation Dataset for Legal Word Embedding: A Case Study On Chinese Codex](https://mlsanity.com/paper/2203.15173/) (2022)

## Abstract (arXiv)

The performance of the Chinese Word Segmentation (CWS) systems has gradually reached a plateau with the rapid development of deep neural networks, especially the successful use of large pre-trained models. In this paper, we take stock of what we have achieved and rethink what's left in the CWS task. Methodologically, we propose a fine-grained evaluation for existing CWS systems, which not only allows us to diagnose the strengths and weaknesses of existing models (under the in-dataset setting), but enables us to quantify the discrepancy between different criterion and alleviate the negative transfer problem when doing multi-criteria learning. Strategically, despite not aiming to propose a novel model in this paper, our comprehensive experiments on eight models and seven datasets, as well as thorough analysis, could search for some promising direction for future research. We make all codes publicly available and release an interface that can quickly evaluate and diagnose user's models: https://github.com/neulab/InterpretEval.