# PRJ: Perception-Retrieval-Judgement for Generated Images

arXiv: https://arxiv.org/abs/2506.03683 · HTML: https://mlsanity.com/paper/2506.03683/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 30 of its 40 references that are in the map.
- First posted on arXiv 4 Jun 2025.

## Builds on (most cited first, 30 of 30)

- [Learning Transferable Visual Models From Natural Language Supervision](https://mlsanity.com/paper/2103.00020/) (2021)
- [GPT-4 Technical Report](https://mlsanity.com/paper/2303.08774/) (2023)
- [The Llama 3 Herd of Models](https://mlsanity.com/paper/2407.21783/) (2024)
- [Visual Instruction Tuning](https://mlsanity.com/paper/2304.08485/) (2023)
- [Improved Baselines with Visual Instruction Tuning](https://mlsanity.com/paper/2310.03744/) (2023)
- [SDXL: Improving Latent Diffusion Models for High-Resolution Image Synthesis](https://mlsanity.com/paper/2307.01952/) (2023)
- [Safe Latent Diffusion: Mitigating Inappropriate Degeneration in Diffusion Models](https://mlsanity.com/paper/2211.05105/) (2022)
- [Adversarial Examples Improve Image Recognition](https://mlsanity.com/paper/1911.09665/) (2019)
- [PoisonedRAG: Knowledge Corruption Attacks to Retrieval-Augmented Generation of Large Language Models](https://mlsanity.com/paper/2402.07867/) (2024)
- [To Generate or Not? Safety-Driven Unlearned Diffusion Models Are Still Easy To Generate Unsafe Images ... For Now](https://mlsanity.com/paper/2310.11868/) (2023)
- [MMA-Diffusion: MultiModal Attack on Diffusion Models](https://mlsanity.com/paper/2311.17516/) (2023)
- [SneakyPrompt: Jailbreaking Text-to-image Generative Models](https://mlsanity.com/paper/2305.12082/) (2023)
- [Can Machines Help Us Answering Question 16 in Datasheets, and In Turn Reflecting on Inappropriate Content?](https://mlsanity.com/paper/2202.06675/) (2022)
- [Jailbreak Vision Language Models via Bi-Modal Adversarial Prompt](https://mlsanity.com/paper/2406.04031/) (2024)
- [A Survey on Retrieval-Augmented Text Generation for Large Language Models](https://mlsanity.com/paper/2404.10981/) (2024)
- [A Pilot Study of Query-Free Adversarial Attack against Stable Diffusion](https://mlsanity.com/paper/2303.16378/) (2023)
- [SafeBench: A Safety Evaluation Framework for Multimodal Large Language Models](https://mlsanity.com/paper/2410.18927/) (2024)
- [Reasoning-Augmented Conversation for Multi-Turn Jailbreak Attacks on Large Language Models](https://mlsanity.com/paper/2502.11054/) (2025)
- [Training Robust Deep Neural Networks via Adversarial Noise Propagation](https://mlsanity.com/paper/1909.09034/) (2019)
- [Manipulating Multimodal Agents via Cross-Modal Prompt Injection](https://mlsanity.com/paper/2504.14348/) (2025)
- [A Comprehensive Evaluation Framework for Deep Model Robustness](https://mlsanity.com/paper/2101.09617/) (2021)
- [Harnessing Perceptual Adversarial Patches for Crowd Counting](https://mlsanity.com/paper/2109.07986/) (2021)
- [Pre-trained Trojan Attacks for Visual Recognition](https://mlsanity.com/paper/2312.15172/) (2023)
- [Unveiling the Safety of GPT-4o: An Empirical Study using Jailbreak Attacks](https://mlsanity.com/paper/2406.06302/) (2024)
- [SurrogatePrompt: Bypassing the Safety Filter of Text-to-Image Models via Substitution](https://mlsanity.com/paper/2309.14122/) (2023)
- [Towards Understanding the Safety Boundaries of DeepSeek Models: Evaluation and Findings](https://mlsanity.com/paper/2503.15092/) (2025)
- [Unlocking the Emotional World of Visual Media: An Overview of the Science, Research, and Impact of Understanding Emotion](https://mlsanity.com/paper/2307.13463/) (2023)
- [CogMorph: Cognitive Morphing Attacks for Text-to-Image Models](https://mlsanity.com/paper/2501.11815/) (2025)
- [DLP: towards active defense against backdoor attacks with decoupled learning process](https://mlsanity.com/paper/2406.13098/) (2024)
- [NBA: defensive distillation for backdoor removal via neural behavior alignment](https://mlsanity.com/paper/2406.10846/) (2024)

## Built on by (most cited first, 2 of 2)

- [Pushing the Limits of Safety: A Technical Report on the ATLAS Challenge 2025](https://mlsanity.com/paper/2506.12430/) (2025)
- [PromptSafe: Gated Prompt Tuning for Safe Text-to-Image Generation](https://mlsanity.com/paper/2508.01272/) (2025)

## Abstract (arXiv)

The rapid progress of generative AI has enabled remarkable creative capabilities, yet it also raises urgent concerns regarding the safety of AI-generated visual content in real-world applications such as content moderation, platform governance, and digital media regulation. This includes unsafe material such as sexually explicit images, violent scenes, hate symbols, propaganda, and unauthorized imitations of copyrighted artworks. Existing image safety systems often rely on rigid category filters and produce binary outputs, lacking the capacity to interpret context or reason about nuanced, adversarially induced forms of harm. In addition, standard evaluation metrics (e.g., attack success rate) fail to capture the semantic severity and dynamic progression of toxicity. To address these limitations, we propose Perception-Retrieval-Judgement (PRJ), a cognitively inspired framework that models toxicity detection as a structured reasoning process. PRJ follows a three-stage design: it first transforms an image into descriptive language (perception), then retrieves external knowledge related to harm categories and traits (retrieval), and finally evaluates toxicity based on legal or normative rules (judgement). This language-centric structure enables the system to detect both explicit and implicit harms with improved interpretability and categorical granularity. In addition, we introduce a dynamic scoring mechanism based on a contextual toxicity risk matrix to quantify harmfulness across different semantic dimensions. Experiments show that PRJ surpasses existing safety checkers in detection accuracy and robustness while uniquely supporting structured category-level toxicity interpretation.