OmniVL-Guard Pro: A Tool-Augmented Agent for Omnibus Vision-Language Forensics
On the map: LLM agents and tool use → Multimodal and embodied agents
Abstract (arXiv)
Existing vision-language forgery detection and grounding methods operate under a closed-world paradigm, assuming verification can be completed by the model alone. However, self-contained MLLMs are constrained by finite parametric knowledge, static training corpora, and limited perceptual resolution, creating a practical ceiling in dynamic open-world forensics -- particularly for real-time event verification requiring external clues and forgery segmentation demanding fine-grained scrutiny of local manipulations. To address these limitations, we shift from scaling up the self-contained model toward reaching beyond it. We propose OmniVL-Guard Pro, a tool-augmented agent that extends unified forensics from closed-world prediction to open-world clues-driven reasoning. OmniVL-Guard Pro integrates a tool environment spanning real-time event search, local cropping and zooming, edge-anomaly screening, face detection, video frame extraction, and SAM3-based segmentation. To generate high-quality tool-reasoning trajectories, we introduce Tree-Structured Self-Evolving Tool Trajectory Generation, which produces diverse trajectories through seed guidance, guider-free self-evolution, and weakly-hinted hard sample synthesis, yielding the Full-Spectrum Tool Reasoning (FSTR) dataset for training. We further propose Checker-Guided Agentic Reinforcement Learning (CGARL), which provides process-level supervision to penalize cases where the answer is correct but the reasoning is distorted. Extensive experiments demonstrate that OmniVL-Guard Pro achieves state-of-the-art performance across various tasks, and exhibits strong zero-shot generalization. The FSTR dataset and code for OmniVL-Guard Pro will be publicly released at https://github.com/shen8424/OmniVL-Guard-Pro.
Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in Multimodal and embodied agents, one of 16 directions in LLM agents and tool use. Builds on 32 of its 42 references that are in the map. First posted on arXiv 16 May 2026.
Builds on
32 of 42 references.
References · 32
- 2024The Llama 3 Herd of Modelscited by 14,715
- 2025Qwen3 Technical Reportcited by 8,820
- 2025Gemini 2.5: Pushing the Frontier with Advanced Reasoning, Multimodality, Long Context, and Next Generation Agentic Capabilitiescited by 3,796
- 2025SAM 3: Segment Anything with Conceptscited by 906
- 2025Video-R1: Reinforcing Video Reasoning in MLLMscited by 484
- 2025Seed1.5-VL Technical Reportcited by 373
- 2020LUKE: Deep Contextualized Entity Representations with Entity-aware Self-attentioncited by 326
- 2021ForgeryNet: A Versatile Benchmark for Comprehensive Forgery Analysiscited by 126
- 2021NewsCLIPpings: Automatic Generation of Out-of-Context Multimodal Mediacited by 107
- 2023Detecting and Grounding Multi-Modal Media Manipulationcited by 99
- 2025Soft Adaptive Policy Optimizationcited by 96
- 2023Generative Image Dynamicscited by 94
20 more
- 2024DeMamba: AI-Generated Video Detection on Million-Scale GenVideo Benchmarkcited by 92
- 2021Open-Domain, Content-based, Multi-modal Fact-checking of Out-of-Context Images via Online Resourcescited by 71
- 2025Spot the Fake: Large Multimodal Model-Based Synthetic Image Detection with Artifact Explanationcited by 68
- 2024MMFakeBench: A Mixed-Source Multimodal Misinformation Detection Benchmark for LVLMscited by 61
- 2024LOKI: A Comprehensive Synthetic Data Detection Benchmark using Large Multimodal Modelscited by 51
- 2024LoRA Land: 310 Fine-tuned LLMs that Rival GPT-4, A Technical Reportcited by 47
- 2023AutoSplice: A Text-prompt Manipulated Image Dataset for Media Forensicscited by 46
- 2024D3: Scaling Up Deepfake Detection by Learning from Discrepancycited by 45
- 2021Masked Face Recognition Challenge: The InsightFace Track Reportcited by 41
- 2022FakeSV: A Multimodal Benchmark with Rich Social Context for Fake News Detection on Short Video Platformscited by 41
- 2024On Learning Multi-Modal Forgery Representation for Diffusion Generated Video Detectioncited by 40
- 2024FKA-Owl: Advancing Multimodal Fake News Detection through Knowledge-Augmented LVLMscited by 38
- 2024FakingRecipe: Detecting Fake News on Short Video Platforms from the Perspective of Creative Processcited by 25
- 2025OpenSDI: Spotting Diffusion-Generated Images in the Open Worldcited by 23
- 2024MCFEND: A Multi-source Benchmark Dataset for Chinese Fake News Detectioncited by 12
- 2025The Coherence Trap: When MLLM-Crafted Narratives Exploit Manipulated Visual Contextscited by 8
- 2025FakeSV-VLM: Taming VLM for Detecting Fake Short-Video News via Progressive Mixture-Of-Experts Adaptercited by 7
- 2025Beyond Artificial Misalignment: Detecting and Grounding Semantic-Coordinated Multimodal Manipulationscited by 4
- 2024Towards Effective Image Forensics via A Novel Computationally Efficient Framework and A New Image Splice Datasetcited by 1
- 2026OmniVL-Guard: Towards Unified Vision-Language Forgery Detection and Grounding via Balanced RLcited by 1
Built on by
Not cited yet.
Similar, not cited
Close in wording, no citation either way
- 2026ForgeryVCR: Visual-Centric Reasoning via Efficient Forensic Tools in MLLMs for Image Forgery Detection and Localization
- 2026Agentic Tool-Augmented Reasoning for Explainable Image Forgery Detection
- 2025REVEAL -- Reasoning and Evaluation of Visual Evidence through Aligned Language
- 2025Code-in-the-Loop Forensics: Agentic Tool Use for Image Forgery Detection
- 2026FACT: A Forensic Agent with Compiled Tool-Use Trajectories for AI-Generated Image Detection
- 2024ForgeryGPT: A Multimodal LLM for Interpretable Image Forgery Detection and Localization
- 2026ForensicsTok: Forensics-Guided Tokenized Modeling for Image Tampering Localization
- 2025Unlocking the Forgery Detection Potential of Vanilla MLLMs: A Novel Training-Free Pipeline