Calling the Bluff: Detecting Ever-Shifting Harmful Chat Dialogue via Ordered Reasoning Chain Regularization
On the map: LLM reasoning → Reasoning benchmarks and evaluation
Abstract (arXiv)
Harmful chat dialogues are ever-shifting through type-shifting and lexical evasion, yet we find they share invariant principles, i.e., an Ordered Reasoning Chain (ORC) of recurring topics, harm language indicators, severity hierarchies, and type characteristics, which can help us capture the key information in the frequently changing lexical expressions. We propose BRACE, which encodes the ORC as four differentiable stages (Topic -> Indicator -> Severity -> Type) with intermediate supervision, serving as a structured regularizer blended with direct heads, and supported by prototype-based feature augmentation and feature path disentanglement. The evaluation results show that, across 4 domains and 5 harm categories, BRACE achieves harm-type macro F1 of 0.934 (RoBERTa-wwm-ext, 3-seed mean), with decoder backbones (Qwen3-1.7B LoRA) reaching 0.949. Ablation studies show that all components contribute to BRACE, and the structural decomposition of ORC enables BRACE to distinguish harmful types with semantic ambiguity. Disclaimer: This paper may contain content that is disturbing to some readers.
Key facts. No citing papers in the map yet (data through 7 Oct 2026). Placed in Reasoning benchmarks and evaluation, one of 17 directions in LLM reasoning. Builds on 9 of its 38 references that are in the map. First posted on arXiv 9 Aug 2026.
Builds on
9 of 38 references.
References · 9
- 2023LMSYS-Chat-1M: A Large-Scale Real-World LLM Conversation Datasetcited by 449
- 2024A Mechanistic Understanding of Alignment Algorithms: A Case Study on DPO and Toxicitycited by 179
- 2023Safety Assessment of Chinese Large Language Modelscited by 82
- 2025ReasoningShield: Safety Detection over Reasoning Traces of Large Reasoning Modelscited by 9
- 2024GW-MoE: Resolving Uncertainty in MoE Router with Global Workspace Theorycited by 8
- 2023JADE: A Linguistics-based Safety Evaluation Platform for Large Language Modelscited by 6
- 2024ChineseSafe: A Chinese Benchmark for Evaluating Safety in Large Language Modelscited by 6
- 2025ChineseHarm-Bench: A Chinese Harmful Content Detection Benchmarkcited by 6
- 2025Toxicity in Online Platforms and AI Systems: A Survey of Needs, Challenges, Mitigations, and Future Directionscited by 3
Built on by
Not cited yet.
Similar, not cited
Close in wording, no citation either way
- 2026RoTRAG: Rule of Thumb Reasoning for Conversation Harm Detection with Retrieval-Augmented Generation
- 2025HarmTransform: Transforming Explicit Harmful Queries into Stealthy via Multi-Agent Debate
- 2025SMARTER: A Data-efficient Framework to Improve Toxicity Detection with Explanation via Self-augmenting Large Language Models
- 2025MemeArena: Automating Context-Aware Unbiased Evaluation of Harmfulness Understanding for Multimodal Large Language Models
- 2022Rethinking Offensive Text Detection as a Multi-Hop Reasoning Problem
- 2025Falcon: A Cross-Modal Evaluation Dataset for Comprehensive Safety Perception
- 2025Refining Input Guardrails: Enhancing LLM-as-a-Judge Efficiency Through Chain-of-Thought Fine-Tuning and Alignment
- 2026HarmThoughts: A Benchmark for Fine-Grained Harmful Behavior Detection in Reasoning Traces