mlsanityData to 14 Oct 2026

SinoGlyphBench: A Diagnostic Benchmark for Chinese Glyph-Level Obfuscation in Language-Model Moderation

arXiv 2609.05843 ↗First posted 5 Sep 2026cs.CLcited by 0

Not on the map. Why?

Abstract (arXiv)

Glyph-level obfuscation can leave harmful Chinese content readable to humans while degrading automated moderation. We introduce SinoGlyphBench, a diagnostic benchmark that identifies label-critical semantic anchors and creates matched original and glyph-obfuscated inputs in text and image modalities. By perturbing anchors, background context, or both, this design distinguishes corruption of moderation-relevant evidence from general surface variation. Across 176,916 paired evaluations of 12 LLMs and MLLMs, obfuscation increases harmful false-negative and false-positive rates by 6.1 and 4.7 percentage points, respectively, and reduces four-way accuracy by 5.0 points. Models retain 75.7% of the decisions that were correct on the matched original inputs. Full-scope perturbations cause the largest degradation, anchor-only perturbations are more damaging than background-only perturbations, and cross-script substitution is particularly difficult in the text modality. Analysis of structured outputs identifies observable mismatches in visible-form reading, intended-message recovery, and final safety judgment. The evaluated models, therefore, remain brittle to Chinese content written with non-canonical glyphs. Resources are available at https://github.com/fengshun124/SinoGlyphBench.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 15 of its 36 references that are in the map. First posted on arXiv 5 Sep 2026.

Builds on

15 of 36 references.

References · 15

  1. 2023FigStep: Jailbreaking Large Vision-Language Models via Typographic Visual Promptscited by 398
  2. 2021Latent Hatred: A Benchmark for Understanding Implicit Hate Speechcited by 217
  3. 2020HateCheck: Functional Tests for Hate Speech Detection Modelscited by 216
  4. 2022COLD: A Benchmark for Chinese Offensive Language Detectioncited by 76
  5. 2022Multilingual HateCheck: Functional Tests for Multilingual Hate Speech Detection Modelscited by 51
  6. 2019Glyce: Glyph-vectors for Chinese Character Representationscited by 46
  7. 2023Facilitating Fine-grained Detection of Chinese Toxic Language: Hierarchical Taxonomy, Resources, and Benchmarkscited by 46
  8. 2024ToxiCloakCN: Evaluating Robustness of Offensive Language Detection in Chinese with Cloaking Perturbationscited by 31
  9. 2026Intern-S1-Pro: Scientific Multimodal Foundation Model at Trillion Scalecited by 29
  10. 2025Exploring Multimodal Challenges in Toxic Chinese Detection: Taxonomy, Benchmark, and Findingscited by 10
  11. 2025Lost in Pronunciation: Detecting Chinese Offensive Language Disguised by Phonetic Cloaking Replacementcited by 10
  12. 2024Silent Signals, Loud Impact: LLMs for Word-Sense Disambiguation of Coded Dog Whistlescited by 7
3 more
  1. 2025ChineseHarm-Bench: A Chinese Harmful Content Detection Benchmarkcited by 7
  2. 2024Making FETCH! Happen: Finding Emergent Dog Whistles Through Common Habitatscited by 6
  3. 2025IYKYK: Using language models to decode extremist cryptolectscited by 3

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2025GlyphMastero: A Glyph Encoder for High-Fidelity Scene Text Editing
  2. 2026Enhancing Multimodal Large Language Models for Ancient Chinese Character Evolution Analysis via Glyph-Driven Fine-Tuning
  3. 2025UniGlyph: Unified Segmentation-Conditioned Diffusion for Precise Visual Text Synthesis
  4. 2026Harder to Defend: Towards Chinese Toxicity Attacks via Implicit Enhancement and Obfuscation Rewriting
  5. 2023GlyphDraw: Seamlessly Rendering Text with Intricate Spatial Structures in Text-to-Image Generation
  6. 2026How China-Origin Vision-Language Models Move from Refusal to Reframing in State Alignment
  7. 2025Information Suppression in Large Language Models: Auditing, Quantifying, and Characterizing Censorship in DeepSeek
  8. 2025Vision Language Models Are Not (Yet) Spelling Correctors