Multi-Modal Agents for Power Distribution Defect Detection: An Evaluation of Foundation Models
On the map: LLM agents and tool use → Financial trading and forecasting agents
Abstract (arXiv)
The power distribution network is critical to reliable electricity delivery, yet traditional inspection methods face limitations in semantic understanding, generalization, and closed-loop automation. To address these challenges, this paper proposes a Multi-Modal Agent framework specifically for power distribution defect detection. Central to this study is the systematic evaluation of multimodal foundation models as unified cognitive engines. We rigorously assess their integrated performance across three critical capabilities: (1) Perception, where the model must accurately identify equipment and generate expert-level descriptions of defects; (2) Reasoning, where the model interprets visual findings to diagnose causes, assess severity, and plan maintenance strategies based on domain knowledge; and (3) Tool Usage, where the model acts as an autonomous operator to execute actions -- such as querying knowledge bases or generating work orders -- to achieve closed-loop maintenance. To support this evaluation, a domain-specific evaluation dataset and a comprehensive benchmark are developed. Experimental results demonstrate the strengths and limitations of current foundation models in these three dimensions, providing empirical evidence for deploying autonomous agents in high-stakes industrial environments.
Key facts. No citing papers in the map yet (data through 7 Oct 2026). Placed in Financial trading and forecasting agents, one of 16 directions in LLM agents and tool use. Builds on 39 of its 43 references that are in the map. First posted on arXiv 11 Jun 2026.
Builds on
39 of 43 references.
Closest ancestors · 1
Same direction, newest first
Other references · 38
- 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
- 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,302
- 2023Judging LLM-as-a-Judge with MT-Bench and Chatbot Arenacited by 8,393
- 2022ReAct: Synergizing Reasoning and Acting in Language Modelscited by 8,355
- 2021Evaluating Large Language Models Trained on Codecited by 8,053
- 2023Visual Instruction Tuningcited by 7,792
- 2023BLIP-2: Bootstrapping Language-Image Pre-training with Frozen Image Encoders and Large Language Modelscited by 6,069
- 2022Flamingo: a Visual Language Model for Few-Shot Learningcited by 4,337
- 2022BLIP: Bootstrapping Language-Image Pre-training for Unified Vision-Language Understanding and Generationcited by 4,310
- 2024Qwen2-VL: Enhancing Vision-Language Model's Perception of the World at Any Resolutioncited by 4,151
- 2023Toolformer: Language Models Can Teach Themselves to Use Toolscited by 4,072
- 2021Scaling Up Visual and Vision-Language Representation Learning With Noisy Text Supervisioncited by 3,848
26 more
- 2023Let's Verify Step by Stepcited by 3,598
- 2023Generative Agents: Interactive Simulacra of Human Behaviorcited by 3,500
- 2023InstructBLIP: Towards General-purpose Vision-Language Models with Instruction Tuningcited by 2,855
- 2023MiniGPT-4: Enhancing Vision-Language Understanding with Advanced Large Language Modelscited by 2,689
- 2023A Survey on Large Language Model based Autonomous Agentscited by 2,203
- 2023MMMU: A Massive Multi-discipline Multimodal Understanding and Reasoning Benchmark for Expert AGIcited by 2,044
- 2023Evaluating Object Hallucination in Large Vision-Language Modelscited by 1,696
- 2023The Rise and Potential of Large Language Model Based Agents: A Surveycited by 1,422
- 2023LLaVA-Med: Training a Large Language-and-Vision Assistant for Biomedicine in One Daycited by 1,287
- 2023AgentBench: Evaluating LLMs as Agentscited by 1,211
- 2023Mind2Web: Towards a Generalist Agent for the Webcited by 1,210
- 2023GAIA: a benchmark for General AI Assistantscited by 1,084
- 2018Object Hallucination in Image Captioningcited by 661
- 2023DriveLM: Driving with Graph Visual Question Answeringcited by 527
- 2023Ego-Exo4D: Understanding Skilled Human Activity from First- and Third-Person Perspectivescited by 477
- 2023Tool Learning with Foundation Modelscited by 302
- 2025Survey on Evaluation of LLM-based Agentscited by 190
- 2025Evaluation and Benchmarking of LLM Agents: A Surveycited by 145
- 2024Large Multimodal Agents: A Surveycited by 99
- 2026Hindsight Credit Assignment for Long-Horizon LLM Agentscited by 51
- 2026Mind the Sim2Real Gap in User Simulation for Agentic Taskscited by 44
- 2026AgentVista: Evaluating Multimodal Agents in Ultra-Challenging Realistic Visual Scenarioscited by 22
- 2024Autonomous Industrial Control using an Agentic Framework with Large Language Modelscited by 19
- 2025Speculative Prefill: Turbocharging TTFT with Lightweight and Training-Free Token Importance Estimationcited by 12
- 2021Ensemble of MRR and NDCG models for Visual Dialogcited by 5
- 2025Large Language Models integration in Smart Gridscited by 3
Built on by
Not cited yet.
Similar, not cited
Close in wording, no citation either way
- 2025Agentic AI Systems in Electrical Power Systems Engineering: Current State-of-the-Art and Challenges
- 2026Power Systems Agent Benchmark: Executable Evaluation of AI Agents in Electric Power Engineering
- 2025Fault2Flow: An AlphaEvolve-Optimized Human-in-the-Loop Multi-Agent System for Fault-to-Workflow Automation
- 2026PowerBench: A Benchmark for Agentic Retrieval and Reasoning in Power Systems
- 2025PowerChain: A Verifiable Agentic AI System for Automating Distribution Grid Analyses
- 2025Grid-Agent: An LLM-Powered Multi-Agent System for Power Grid Control
- 2026RestoreBench: Can AI Agents Restore Power Flow Convergence?
- 2025From Transformers to Large Language Models: A systematic review of AI applications in the energy sector towards Agentic Digital Twins