mlsanityData to 7 Oct 2026

Task-Conditional Faithfulness Auditing of Multimodal LLMs for Grid Diagnosis

arXiv 2607.24539 ↗First posted 27 Jul 2026cs.AI cs.SY eess.SYcited by 0

On the map: Multimodal LLMs / vision-language models → MLLM training and visual instruction tuning

Abstract (arXiv)

Multimodal large language models (LLMs) can combine topology, measurements, and incident text for grid diagnosis, yet answer accuracy does not establish that task-appropriate evidence was used. This letter proposes a general framework in order to conduct task-conditional faithfulness audit. It compares self-reported reliance, intervention-derived behavioral reliance, and preregistered engineering importance. The framework first registers task-specific evidence requirements and compares them with self-reported reliance and behavioral changes under controlled modality ablations. To resolve detected discrepancies, we design an evidence-gated correction and re-audit mechanism that regenerates failed responses under evidence constraints and independently re-ablates them to verify improved grounding without performance loss. Case studies evaluate three differently scaled LLMs on IEEE 39- and 118-bus scenarios. These results validate the framework ability to detect, diagnose, and correct task-conditional faithfulness failures.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Placed in MLLM training and visual instruction tuning, one of 16 directions in Multimodal LLMs / vision-language models. Builds on 4 of its 7 references that are in the map. First posted on arXiv 27 Jul 2026.

Builds on

4 of 7 references.

Closest ancestors · 1

Same direction, newest first

  1. 2024Quantifying and Mitigating Unimodal Biases in Multimodal Large Language Models: A Causal Perspective27 Mar

Other references · 3

  1. 2023Language Models Don't Always Say What They Think: Unfaithful Explanations in Chain-of-Thought Promptingcited by 1,081
  2. 2025Large Language Models for Power System Security: A Novel Multi-Modal Approach for Anomaly Detection in Energy Management Systemscited by 1
  3. 2025Evaluation of Large Language Models for Numeric Anomaly Detection in Power Systemscited by 1

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2025Differences That Matter: Auditing Models for Capability Gap Discovery and Rectification
  2. 2024Fault Diagnosis in Power Grids with Large Language Model
  3. 2026Targeted Tests for LLM Reasoning: An Audit-Constrained Protocol
  4. 2026The Calls are Coming from Inside the Model: Investigating Probe-based Detection of Tool-Calling Errors in LLMs
  5. 2025Hidden in Plain Sight: Reasoning in Underspecified and Misspecified Scenarios for Multimodal LLMs
  6. 2026MonitorBench: A Comprehensive Benchmark for Chain-of-Thought Monitorability in Large Language Models
  7. 2026Reducing Hallucinations in Large Language Models Through Integrated Self-Verification and Retrieval-Augmented Generation
  8. 2025Can LLMs Detect Their Confabulations? Estimating Reliability in Uncertainty-Aware Language Models