# Towards Evaluating the Robustness of Chinese BERT Classifiers

arXiv: https://arxiv.org/abs/2004.03742 · HTML: https://mlsanity.com/paper/2004.03742/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 7 Apr 2020.

## Built on by (most cited first, 2 of 2)

- [Expanding Scope: Adapting English Adversarial Attacks to Chinese](https://mlsanity.com/paper/2306.04874/) (2023)
- [Is Multi-Modal Necessarily Better? Robustness Evaluation of Multi-modal Fake News Detection](https://mlsanity.com/paper/2206.08788/) (2022)

## Abstract (arXiv)

Recent advances in large-scale language representation models such as BERT have improved the state-of-the-art performances in many NLP tasks. Meanwhile, character-level Chinese NLP models, including BERT for Chinese, have also demonstrated that they can outperform the existing models. In this paper, we show that, however, such BERT-based models are vulnerable under character-level adversarial attacks. We propose a novel Chinese char-level attack method against BERT-based classifiers. Essentially, we generate "small" perturbation on the character level in the embedding space and guide the character substitution procedure. Extensive experiments show that the classification accuracy on a Chinese news dataset drops from 91.8% to 0% by manipulating less than 2 characters on average based on the proposed attack. Human evaluations also confirm that our generated Chinese adversarial examples barely affect human performance on these NLP tasks.