mlsanityData to 7 Oct 2026

RiskChainBench: A Benchmark for Obfuscated Platform Message Restoration and Evidence-Grounded Web Investigation

arXiv 2609.16900 ↗First posted 15 Sep 2026cs.CLcited by 0

Not on the map. Why?

Abstract (arXiv)

Platform abuse campaigns conceal redirection instructions with emojis, homophones, character decomposition, and redundant symbols, then route users through disguised links to services associated with pornography, fraud, gambling, or illicit transactions. Existing benchmarks evaluate obfuscated text and risky webpages separately, obscuring how target recovery affects downstream evidence acquisition. We introduce RiskChainBench, pairing 3,600 synthetic token-text restoration inputs from 600 source sessions with 600 corresponding human-labeled local web environments. A model first restores the message, operational intent, and destination; the same underlying model then acts as a VLM-driven web agent that investigates the correctly associated website and produces a frozen, evidence-cited risk report without message-side semantics or domain-reputation cues. We score restoration and correct-routing web investigation separately and compose them offline by applying the frozen primary-entry prediction as a gate to the same Task 2 result. Human labels determine task correctness, while a fixed multimodal evidence judge assesses faithfulness, sufficiency, completeness, and consistency. Across ten models, Entry Top-1 ranges from 35.2% to 95.2% and web decision accuracy from 26.3% to 62.8%; the leading systems differ across entry recovery, full reconstruction, website decisions, and fine-grained typing. Execution failures account for 31.9% of web runs, whereas post-decision type errors account for only 0.9%, identifying stable exploration and risk judgment as the principal bottlenecks. We release the benchmark, protocol, and resettable local sandbox.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 8 of its 20 references that are in the map. First posted on arXiv 15 Sep 2026.

Builds on

8 of 20 references.

References · 8

  1. 2023WebArena: A Realistic Web Environment for Building Autonomous Agentscited by 1,908
  2. 2025AgentRewardBench: Evaluating Automatic Evaluations of Web Agent Trajectoriescited by 99
  3. 2021Hatemoji: A Test Suite and Adversarially-Generated Dataset for Benchmarking and Detecting Emoji-based Hatecited by 50
  4. 2025Lost in Pronunciation: Detecting Chinese Offensive Language Disguised by Phonetic Cloaking Replacementcited by 10
  5. 2026Time to REFLECT: Can We Trust LLM Judges for Evidence-based Research Agents?cited by 10
  6. 2026Aligning Agents via Planning: A Benchmark for Trajectory-Level Reward Modelingcited by 7
  7. 2026"Newspaper Eat" Means "Not Tasty": A Taxonomy and Benchmark for Coded Language in Real-World Chinese Online Reviewscited by 3
  8. 2026MalURLBench: A Benchmark Evaluating Agents' Vulnerabilities When Processing Web URLscited by 1

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2026Evaluating Deep-Search Agents under Hierarchical Web Evidence Poisoning
  2. 2026FraudSMSWalker: Benchmarking Agentic Large Language Models for SMS-to-Webpage Fraud Detection
  3. 2026How Much Can We Trust LLM Search Agents? Measuring Endorsement Vulnerability to Web Content Manipulation
  4. 2026Retrieval Collapses When AI Pollutes the Web
  5. 2026Hidden-in-Plain-Text: A Benchmark for Social-Web Indirect Prompt Injection in RAG
  6. 2024On the Risk of Evidence Pollution for Malicious Social Text Detection in the Era of LLMs
  7. 2025Erased but Not Forgotten: How Backdoors Compromise Concept Erasure
  8. 2026AgentDoxx: Agentic Re-identification of Anonymized Text with Web Search