IndusAgent: Reinforcing Open-Vocabulary Industrial Anomaly Detection with Agentic Tools
On the map: Multimodal LLMs / vision-language models → Chart understanding and anomaly detection
also in 1 more
Abstract (arXiv)
Multimodal large language models (MLLMs) have shown remarkable capability in bridging visual perception and textual reasoning, enabling zero-shot understanding across diverse industrial scenarios. However, their performance in open-vocabulary industrial anomaly detection (IAD) is often limited by domain-misaligned reasoning and hallucinated structural inferences. To address these challenges, we propose IndusAgent, a tool-augmented agentic framework for open-vocabulary IAD. Specifically, we first construct Indus-CoT, a structured dataset that integrates global visual observations, high-resolution local patches, and expert normalcy priors, providing supervision for fine-tuning the model on rigorous industrial inspection trajectories. Building on this, IndusAgent dynamically orchestrates a set of external tools, including dynamic region cropping, high-frequency feature enhancement, and prior retrieval, thus enabling the agent to actively resolve visual ambiguities and disentangle subtle anomalies. Furthermore, we introduce a gated reinforcement learning objective that jointly optimizes anomaly classification, localization accuracy, anomaly type reasoning, and efficient tool usage, ensuring that tool invocation occurs only when beneficial. Extensive evaluations on five industrial anomaly benchmarks, including MVTec-AD, VisA, MPDD, DTD, and SDD, demonstrate that IndusAgent achieves state-of-the-art zero-shot performance among all existing methods, validating our robustness and generalization capacity.
Key facts. Cited by 1 paper in the map (data through 13 Oct 2026). Placed in Chart understanding and anomaly detection, one of 16 directions in Multimodal LLMs / vision-language models. Builds on 77 of its 122 references that are in the map. First posted on arXiv 20 May 2026.
Builds on
77 of 122 references.
Closest ancestors · 2
Same direction, newest first
- 2025Towards Zero-Shot Anomaly Detection and Reasoning with Multimodal Large Language Models11 Feb
- 2023AnomalyGPT: Detecting Industrial Anomalies Using Large Vision-Language Models29 Aug
Other references · 75
- 2022Training language models to follow instructions with human feedbackcited by 15,856
- 2017Proximal Policy Optimization Algorithmscited by 14,592
- 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,376
- 2024DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Modelscited by 9,091
- 2023Visual Instruction Tuningcited by 7,827
- 2025Qwen2.5-VL Technical Reportcited by 5,499
- 2023Improved Baselines with Visual Instruction Tuningcited by 4,389
- 2023Toolformer: Language Models Can Teach Themselves to Use Toolscited by 4,117
- 2022Self-Instruct: Aligning Language Models with Self-Generated Instructionscited by 2,693
- 2024LLaVA-OneVision: Easy Visual Task Transfercited by 2,692
- 2025Qwen3-VL Technical Reportcited by 2,663
- 2023PaLM-E: An Embodied Multimodal Language Modelcited by 2,052
38 more
- 2022Visual Prompt Tuningcited by 1,677
- 2024How Far Are We to GPT-4V? Closing the Gap to Commercial Multimodal Models with Open-Source Suitescited by 1,041
- 2021Towards Total Recall in Industrial Anomaly Detectioncited by 774
- 2023Multimodal Chain-of-Thought Reasoning in Language Modelscited by 746
- 2023Aligning Large Multimodal Models with Factually Augmented RLHFcited by 708
- 2020PaDiM: a Patch Distribution Modeling Framework for Anomaly Detection and Localizationcited by 593
- 2021DRAEM -- A discriminatively trained reconstruction embedding for surface anomaly detectioncited by 511
- 2022Visual Programming: Compositional visual reasoning without trainingcited by 510
- 2025Video-R1: Reinforcing Video Reasoning in MLLMscited by 466
- 2022SPot-the-Difference Self-Supervised Pre-training for Anomaly Detection and Segmentationcited by 399
- 2023AgentTuning: Enabling Generalized Agent Abilities for LLMscited by 325
- 2023WinCLIP: Zero-/Few-Shot Anomaly Classification and Segmentationcited by 314
- 2018Improving Unsupervised Defect Segmentation by Applying Structural Similarity to Autoencoderscited by 264
- 2023FireAct: Toward Language Agent Fine-tuningcited by 250
- 2025LMM-R1: Empowering 3B LMMs with Strong Reasoning Abilities Through Two-Stage Rule-Based RLcited by 246
- 2021FastFlow: Unsupervised Anomaly Detection and Localization via 2D Normalizing Flowscited by 226
- 2025Imagine while Reasoning in Space: Multimodal Visualization-of-Thoughtcited by 205
- 2023LLaVA-Plus: Learning to Use Tools for Creating Multimodal Agentscited by 191
- 2021Fully Convolutional Cross-Scale-Flows for Image-based Defect Detectioncited by 117
- 2025PyVision: Agentic Vision with Dynamic Toolingcited by 73
- 2023Visual Program Distillation: Distilling Tools and Programmatic Reasoning into Vision-Language Modelscited by 66
- 2024Change-Agent: Towards Interactive Comprehensive Remote Sensing Change Interpretation and Analysiscited by 61
- 2025PRISM: Self-Pruning Intrinsic Selection Method for Training-Free Multimodal Data Selectioncited by 59
- 2024CRS-Diff: Controllable Remote Sensing Image Generation with Diffusion Modelcited by 54
- 2025CoT-Kinetics: A Theoretical Modeling Assessing LRM Reasoning Processcited by 51
- 2024Context-DPO: Aligning Language Models for Context-Faithfulnesscited by 50
- 2025Pure Vision Language Action (VLA) Models: A Comprehensive Surveycited by 47
- 2025Text2Earth: Unlocking Text-driven Remote Sensing Image Generation with a Global-Scale Dataset and a Foundation Modelcited by 46
- 2025Parameters vs. Context: Fine-Grained Control of Knowledge Reliance in Language Modelscited by 38
- 2020Cross-View Tracking for Multi-Human 3D Pose Estimation at over 100 FPScited by 36
- 2024Decoding by Contrasting Knowledge: Enhancing LLMs' Confidence on Edited Factscited by 36
- 2024AeroGen: Enhancing Remote Sensing Object Detection with Diffusion-Driven Data Generationcited by 32
- 2025AnomalyR1: A GRPO-based End-to-end MLLM for Industrial Anomaly Detectioncited by 28
- 2025DecAlign: Hierarchical Cross-Modal Alignment for Decoupled Multimodal Representation Learningcited by 24
- 2025Video-STAR: Reinforcing Open-Vocabulary Action Recognition with Toolscited by 24
- 2026HABIT: Chrono-Synergia Robust Progressive Learning Framework for Composed Image Retrievalcited by 24
- 2025FingER: Content Aware Fine-grained Evaluation with Reasoning for AI-Generated Videoscited by 22
- 2025From Editor to Dense Geometry Estimatorcited by 22
50 most cited of 75 All 75 (JSON)
Built on by
Citing papers per year: 2026: 1.
1 paper.
Similar, not cited
Close in wording, no citation either way
- 2026Is In-Domain Training Enough for Fine-Grained Industrial Anomaly Understanding?
- 2026Industrial Anomaly Detection via Defect-Grounded Reasoning in Visual Latent Space
- 2026A Structured Benchmark for Text-Guided Anomaly Detection: When Language Stops Conditioning the Decision
- 2025Can Multimodal Large Language Models be Guided to Improve Industrial Anomaly Detection?
- 2026Plan First, Judge Later, Run Better: A DMAIC-Inspired Agentic System for Industrial Anomaly Detection
- 2026AnomalyAgent: Training-Free Agentic Models for Zero-/Few-Shot Anomaly Detection
- 2024VMAD: Visual-enhanced Multimodal Large Language Model for Zero-Shot Anomaly Detection
- 2026Towards Explainable Industrial Anomaly Detection via Knowledge-Guided Latent Reasoning