mlsanityData to 14 Oct 2026

Beyond Balanced Accuracy: A Resolution and Parity-Controlled Benchmark for Vision-Language and Vision-Only Defect Assessment in UAV Power-Line Inspection

arXiv 2609.27457 ↗First posted 23 Sep 2026cs.CVcited by 0

On the map: Multimodal LLMs / vision-language models → Chart understanding and anomaly detection also fits: VLM benchmarks and visual reasoning

Abstract (arXiv)

Vision-language models (VLMs) are often reported to outperform task-specific vision backbones for unmanned aerial vehicle (UAV) power-line defect assessment. We test that claim on ElecVQA-Bench, a 56,972-item benchmark derived from the public InsPLAD dataset, across six evaluation choices: partition, evaluated item set, label space, replication, input resolution, and side information. On a matched partition, a Swin Transformer and the strongest adapted VLM differ by only 0.03 points at binary screening. At seven-way defect typing, increasing the vision backbones from 224 px to the measured pixel budget of the VLM preprocessor narrows the gap against InternVL3.5-8B from +20.53 to -0.57 points for ResNet-50 and from +23.67 to +4.70 points for Swin-T. A pixel-budget audit shifts Qwen3-VL-8B macro recall by 10.78 points, yet a source-pixel-matched InternVL control still leaves Qwen ahead by 7.43 to 13.61 points while using 56% fewer visual tokens, so neither source pixels nor token budget explains the difference between the two VLMs. A two-seed global replication changes Qwen binary accuracy and seven-way macro recall by 0.86 and 1.02 points. After split-specific retraining, Qwen does not lead at crop or image level, and a 14-tower, three-seed replication reverses the sign across seeds, giving mean common-six macro recall of 0.9085 for Qwen against 0.9509 for ResNet-50. No split regime yields a family-level advantage that survives multiple-comparison correction. The study supports a benchmark-audit contribution rather than a general claim of VLM superiority.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in Chart understanding and anomaly detection, one of 16 directions in Multimodal LLMs / vision-language models. Builds on 22 of its 40 references that are in the map. First posted on arXiv 23 Sep 2026.

Builds on

22 of 40 references.

References · 22

  1. 2015Deep Residual Learning for Image Recognitioncited by 52,096
  2. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,744
  3. 2021Swin Transformer: Hierarchical Vision Transformer using Shifted Windowscited by 12,139
  4. 2015Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networkscited by 9,639
  5. 2023Visual Instruction Tuningcited by 7,869
  6. 2023BLIP-2: Bootstrapping Language-Image Pre-training with Frozen Image Encoders and Large Language Modelscited by 6,137
  7. 2025Qwen2.5-VL Technical Reportcited by 5,587
  8. 2017On Calibration of Modern Neural Networkscited by 4,960
  9. 2015You Only Look Once: Unified, Real-Time Object Detectioncited by 4,852
  10. 2023QLoRA: Efficient Finetuning of Quantized LLMscited by 3,418
  11. 2023InternVL: Scaling up Vision Foundation Models and Aligning for Generic Visual-Linguistic Taskscited by 2,565
  12. 2023Evaluating Object Hallucination in Large Vision-Language Modelscited by 1,729
10 more
  1. 2019Class-Balanced Loss Based on Effective Number of Samplescited by 1,245
  2. 2019Decoupling Representation and Classifier for Long-Tailed Recognitioncited by 809
  3. 2021Towards Total Recall in Industrial Anomaly Detectioncited by 776
  4. 2018Object Hallucination in Image Captioningcited by 669
  5. 2017A systematic study of the class imbalance problem in convolutional neural networkscited by 513
  6. 2019Which Tasks Should Be Learned Together in Multi-task Learning?cited by 375
  7. 2020Same Same But DifferNet: Semi-Supervised Defect Detection with Normalizing Flowscited by 212
  8. 2023Deep Industrial Image Anomaly Detection: A Surveycited by 148
  9. 2021Fully Convolutional Cross-Scale-Flows for Image-based Defect Detectioncited by 117
  10. 2023InsPLAD: A Dataset and Benchmark for Power Line Asset Inspection in UAV Imagescited by 15

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2026Synthetic Defect Image Generation for Power Line Insulator Inspection Using Multimodal Large Language Models
  2. 20263D-DefectBench: A Controlled Factorial Study of Vision-Language Model Evaluation Pipelines for Fine-Grained 3D Generation Defects
  3. 2024Transmission Line Defect Detection Based on UAV Patrol Images and Vision-language Pretraining
  4. 2025Bridging vision language model (VLM) evaluation gaps with a framework for scalable and cost-effective benchmark generation
  5. 2025Seeing the Unseen: Towards Training-Free Inspection for Wind Turbine Blades Using Knowledge-Augmented Vision Language Models
  6. 2026A Hybrid Vision-Language Architecture for Automated Defect Reasoning and Report Generation in Industrial Inspection
  7. 2025InspectVLM: Unified in Theory, Unreliable in Practice
  8. 2026WildRoadBench: A Wild Aerial Road-Damage Grounding Benchmark for Vision-Language Models and Autonomous Agents