# SinoGlyphBench: A Diagnostic Benchmark for Chinese Glyph-Level Obfuscation in Language-Model Moderation

arXiv: https://arxiv.org/abs/2609.05843 · HTML: https://mlsanity.com/paper/2609.05843/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 15 of its 36 references that are in the map.
- First posted on arXiv 5 Sep 2026.

## Builds on (most cited first, 15 of 15)

- [FigStep: Jailbreaking Large Vision-Language Models via Typographic Visual Prompts](https://mlsanity.com/paper/2311.05608/) (2023)
- [Latent Hatred: A Benchmark for Understanding Implicit Hate Speech](https://mlsanity.com/paper/2109.05322/) (2021)
- [HateCheck: Functional Tests for Hate Speech Detection Models](https://mlsanity.com/paper/2012.15606/) (2020)
- [COLD: A Benchmark for Chinese Offensive Language Detection](https://mlsanity.com/paper/2201.06025/) (2022)
- [Multilingual HateCheck: Functional Tests for Multilingual Hate Speech Detection Models](https://mlsanity.com/paper/2206.09917/) (2022)
- [Glyce: Glyph-vectors for Chinese Character Representations](https://mlsanity.com/paper/1901.10125/) (2019)
- [Facilitating Fine-grained Detection of Chinese Toxic Language: Hierarchical Taxonomy, Resources, and Benchmarks](https://mlsanity.com/paper/2305.04446/) (2023)
- [ToxiCloakCN: Evaluating Robustness of Offensive Language Detection in Chinese with Cloaking Perturbations](https://mlsanity.com/paper/2406.12223/) (2024)
- [Intern-S1-Pro: Scientific Multimodal Foundation Model at Trillion Scale](https://mlsanity.com/paper/2603.25040/) (2026)
- [Exploring Multimodal Challenges in Toxic Chinese Detection: Taxonomy, Benchmark, and Findings](https://mlsanity.com/paper/2505.24341/) (2025)
- [Lost in Pronunciation: Detecting Chinese Offensive Language Disguised by Phonetic Cloaking Replacement](https://mlsanity.com/paper/2507.07640/) (2025)
- [Silent Signals, Loud Impact: LLMs for Word-Sense Disambiguation of Coded Dog Whistles](https://mlsanity.com/paper/2406.06840/) (2024)
- [ChineseHarm-Bench: A Chinese Harmful Content Detection Benchmark](https://mlsanity.com/paper/2506.10960/) (2025)
- [Making FETCH! Happen: Finding Emergent Dog Whistles Through Common Habitats](https://mlsanity.com/paper/2412.12072/) (2024)
- [IYKYK: Using language models to decode extremist cryptolects](https://mlsanity.com/paper/2506.05635/) (2025)

## Abstract (arXiv)

Glyph-level obfuscation can leave harmful Chinese content readable to humans while degrading automated moderation. We introduce SinoGlyphBench, a diagnostic benchmark that identifies label-critical semantic anchors and creates matched original and glyph-obfuscated inputs in text and image modalities. By perturbing anchors, background context, or both, this design distinguishes corruption of moderation-relevant evidence from general surface variation. Across 176,916 paired evaluations of 12 LLMs and MLLMs, obfuscation increases harmful false-negative and false-positive rates by 6.1 and 4.7 percentage points, respectively, and reduces four-way accuracy by 5.0 points. Models retain 75.7% of the decisions that were correct on the matched original inputs. Full-scope perturbations cause the largest degradation, anchor-only perturbations are more damaging than background-only perturbations, and cross-script substitution is particularly difficult in the text modality. Analysis of structured outputs identifies observable mismatches in visible-form reading, intended-message recovery, and final safety judgment. The evaluated models, therefore, remain brittle to Chinese content written with non-canonical glyphs. Resources are available at https://github.com/fengshun124/SinoGlyphBench.