mlsanityData to 14 Oct 2026

StyleBERT: Chinese pretraining by font style information

arXiv 2202.09955 ↗First posted 21 Feb 2022cs.CLcited by 0

Not on the map. Why?

Abstract (arXiv)

With the success of down streaming task using English pre-trained language model, the pre-trained Chinese language model is also necessary to get a better performance of Chinese NLP task. Unlike the English language, Chinese has its special characters such as glyph information. So in this article, we propose the Chinese pre-trained language model StyleBERT which incorporate the following embedding information to enhance the savvy of language model, such as word, pinyin, five stroke and chaizi. The experiments show that the model achieves well performances on a wide range of Chinese NLP tasks.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 24 of its 34 references that are in the map. First posted on arXiv 21 Feb 2022.

Builds on

24 of 34 references.

References · 24

  1. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,399
  2. 2019RoBERTa: A Robustly Optimized BERT Pretraining Approachcited by 13,372
  3. 2019Sentence-BERT: Sentence Embeddings using Siamese BERT-Networkscited by 9,170
  4. 2016Google's Neural Machine Translation System: Bridging the Gap between Human and Machine Translationcited by 1,475
  5. 2018XNLI: Evaluating Cross-lingual Sentence Representationscited by 915
  6. 2019Text Summarization with Pretrained Encoderscited by 556
  7. 2019How to Fine-Tune BERT for Text Classification?cited by 411
  8. 2019ERNIE: Enhanced Representation through Knowledge Integrationcited by 302
  9. 2020Revisiting Pre-Trained Models for Chinese Natural Language Processingcited by 266
  10. 2019ERNIE 2.0: A Continual Pre-training Framework for Language Understandingcited by 231
  11. 2019BERT Post-Training for Review Reading Comprehension and Aspect-based Sentiment Analysiscited by 176
  12. 2019Fine-tune BERT for Extractive Summarizationcited by 122
12 more
  1. 2019Multi-passage BERT: A Globally Normalized BERT Model for Open-domain Question Answeringcited by 96
  2. 2018A Span-Extraction Dataset for Chinese Machine Reading Comprehensioncited by 84
  3. 2019BERT with History Answer Embedding for Conversational Question Answeringcited by 67
  4. 2021ChineseBERT: Chinese Pretraining Enhanced by Glyph and Pinyin Informationcited by 65
  5. 2019Glyce: Glyph-vectors for Chinese Character Representationscited by 46
  6. 2017Glyph-aware Embedding of Chinese Characterscited by 14
  7. 2020Towards Interpreting BERT for Reading Comprehension Based QAcited by 13
  8. 2017Learning Character-level Compositionality with Visual Featurescited by 12
  9. 2014Radical-Enhanced Chinese Character Embeddingcited by 9
  10. 2019BERT Meets Chinese Word Segmentationcited by 4
  11. 2019Chinese Embedding via Stroke and Glyph Information: A Dual-channel Viewcited by 3
  12. 2021MFE-NER: Multi-feature Fusion Embedding for Chinese Named Entity Recognitioncited by 3

Built on by

Not cited yet.