mlsanityData to 14 Oct 2026

Open-Domain Safety Policy Construction

arXiv 2604.01354 ↗First posted 1 Apr 2026cs.CLcited by 0

Not on the map. Why?

Abstract (arXiv)

Moderation layers are increasingly a core component of many products built on user- or model-generated content. However, drafting and maintaining domain-specific safety policies remains costly. We present Deep Policy Research (DPR), a minimal agentic system that drafts a full content moderation policy based on only human-written seed domain information. DPR uses a single web search tool and lightweight scaffolding to iteratively propose search queries, distill diverse web sources into policy rules, and organize rules into an indexed document. We evaluate DPR on (1) the OpenAI undesired content benchmark across five domains with two compact reader LLMs and (2) an in-house multimodal advertisement moderation benchmark. DPR consistently outperforms definition-only and in-context learning baselines, and in our end-to-end setting it is competitive with expert-written policy sections in several domains. Moreover, under the same seed specification and evaluation protocol, DPR outperforms a general-purpose deep research system, suggesting that a task-specific, structured research loop can be more effective than generic web research for policy drafting. We release our experiment code at https://github.com/xiaowu0162/deep-policy-research.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 7 of its 10 references that are in the map. First posted on arXiv 1 Apr 2026.

Builds on

7 of 10 references.

References · 7

  1. 2024Qwen2.5 Technical Reportcited by 4,942
  2. 2022Constitutional AI: Harmlessness from AI Feedbackcited by 3,050
  3. 2022A Holistic Approach to Undesired Content Detection in the Real Worldcited by 356
  4. 2024Deliberative Alignment: Reasoning Enables Safer Language Modelscited by 308
  5. 2020Directions in Abusive Language Training Data: Garbage In, Garbage Outcited by 140
  6. 2024Rule Based Rewards for Language Model Safetycited by 108
  7. 2021Towards generalisable hate speech detection: a review on obstacles and solutionscited by 66

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2025Policy-as-Prompt: Rethinking Content Moderation in the Age of Large Language Models
  2. 2025Towards Trustworthy Multimodal Moderation via Policy-Aligned Reasoning and Hierarchical Labeling
  3. 2025PAM: Training Policy-Aligned Moderation Filters at Scale
  4. 2026Can Foundation Models Moderate Online Content? Evaluating Instruction- vs. Example-Driven Policy Operationalization
  5. 2026CourtGuard: A Model-Agnostic Framework for Zero-Shot Policy Adaptation in LLM Safety
  6. 2026PolicyAlign: Direct Policy-Based Safety Alignment for Large Language Models
  7. 2026Benchmarking Open-Source Safety Guard Models: A Comprehensive Evaluation
  8. 2025Scaling Reinforcement Learning for Content Moderation with Large Language Models