VARM-Bench: Benchmarking Verifiable Structured Reasoning in Chinese Abusive Speech Moderation
On the map: LLM reasoning → Reasoning benchmarks and evaluation
Abstract (arXiv)
The widespread circulation of abusive online content has increased the need for reliable moderation of Chinese social-media text. Existing Chinese benchmarks support label classification, fine-grained toxicity categorization, and target-aware extraction, but do not provide a unified representation for deterministically verifying the stated basis of a moderation decision. We introduce VARM-Bench, a benchmark for field-anchored chain-of-thought rationales in Chinese abusive-speech moderation. Each instance contains a concise natural-language rationale with explicit anchors for six decisions: target, target type, target explicitness, author stance, harmfulness label, and fine-grained category. Our deterministic protocol evaluates field correctness, target alignment, output validity, complete-record agreement, and hidden record errors conditioned on correct final decisions, without relying on an LLM judge. Under a common structured-output protocol, we evaluate language models across multiple model families using zero-shot prompting, taxonomy guidance, and structured CoT supervision, and analyze lexical-cue sensitivity and field-level errors. Results show that strong label-level performance can conceal substantial errors in complete moderation records. VARM-Bench provides an auditable and reproducible benchmark for evaluating verifiable moderation rationales in Chinese abusive-speech moderation.
Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in Reasoning benchmarks and evaluation, one of 17 directions in LLM reasoning. Builds on 6 of its 31 references that are in the map. First posted on arXiv 16 Aug 2026.
Builds on
6 of 31 references.
References · 6
- 2023Judging LLM-as-a-Judge with MT-Bench and Chatbot Arenacited by 8,626
- 2023Measuring Faithfulness in Chain-of-Thought Reasoningcited by 703
- 2022Towards Identifying Social Bias in Dialog Systems: Frame, Datasets, and Benchmarkscited by 36
- 2023Toxicity Detection is NOT all you Need: Measuring the Gaps to Supporting Volunteer Content Moderatorscited by 10
- 2025ChineseHarm-Bench: A Chinese Harmful Content Detection Benchmarkcited by 7
- 2023How We Define Harm Impacts Data Annotations: Explaining How Annotators Distinguish Hateful, Offensive, and Toxic Commentscited by 2
Built on by
Not cited yet.
Similar, not cited
Close in wording, no citation either way
- 2025SMARTER: A Data-efficient Framework to Improve Toxicity Detection with Explanation via Self-augmenting Large Language Models
- 2024LionGuard: Building a Contextualized Moderation Classifier to Tackle Localized Unsafe Content
- 2026Beyond English and Evasion: A Human-Annotated Multi-Domain Benchmark for High-Stakes LLM Safety Evaluation in Chinese
- 2026From Triage to Impact: A Multi-Hop Reasoning Benchmark for Explainable Persian Hate Speech
- 2025Soft Inductive Bias Approach via Explicit Reasoning Perspectives in Inappropriate Utterance Detection Using Large Language Models
- 2026Harder to Defend: Towards Chinese Toxicity Attacks via Implicit Enhancement and Obfuscation Rewriting
- 2025Chinese Toxic Language Mitigation via Sentiment Polarity Consistent Rewrites
- 2025MemeArena: Automating Context-Aware Unbiased Evaluation of Harmfulness Understanding for Multimodal Large Language Models