YOLO with Kolmogorov-Arnold networks and vision-language foundation models for interpretable object detection with trustworthy multimodal AI in computer vision perception
On the map: Multimodal LLMs / vision-language models → Driving VLMs and open-vocabulary detection
Abstract (arXiv)
The trustworthy object detection capabilities of a novel Kolmogorov-Arnold network framework are examined here. The approach addresses a key limitation in computer vision for vehicle detection perception, and beyond. These systems offer limited transparency regarding the reliability of their confidence scores in visually degraded or ambiguous scenes. To this end, a Kolmogorov-Arnold network is employed as an interpretable post-hoc surrogate to model the trustworthiness of the You Only Look Once (Yolov10) detections using seven geometric and semantic features. The additive spline-based structure of the Kolmogorov-Arnold network enables direct visualisation of each feature's influence. This produces smooth and transparent functional mappings that reveal when the model's confidence is well supported and when it is unreliable. Furthermore, a bootstrapped language-image (BLIP) foundation model generates descriptive captions of each scene. This tool enables a lightweight multimodal interface without affecting the interpretability layer. Experiments on both Common Objects in Context (COCO), and images from the University of Bath campus demonstrate that the framework accurately identifies low-trust predictions under blur, occlusion, or low texture. This provides actionable insights for acceptance, review, or downstream risk mitigation. The resulting system delivers interpretable object detection with trustworthy confidence estimates. It offers a powerful tool for transparent and practical perception component for autonomous and multimodal artificial intelligence applications.
Key facts. No citing papers in the map yet (data through 7 Oct 2026). Placed in Driving VLMs and open-vocabulary detection, one of 16 directions in Multimodal LLMs / vision-language models. Builds on 32 of its 115 references that are in the map. First posted on arXiv 24 Mar 2026.
Builds on
32 of 115 references.
References · 32
- 2024The Llama 3 Herd of Modelscited by 14,192
- 2015You Only Look Once: Unified, Real-Time Object Detectioncited by 4,829
- 2023InternVL: Scaling up Vision Foundation Models and Aligning for Generic Visual-Linguistic Taskscited by 2,534
- 2021Learning to Prompt for Vision-Language Modelscited by 2,269
- 2021YOLOX: Exceeding YOLO Series in 2021cited by 1,064
- 2019EfficientDet: Scalable and Efficient Object Detectioncited by 1,018
- 2018Object Detection with Deep Learning: A Reviewcited by 524
- 2024YOLOv10: Real-Time End-to-End Object Detectioncited by 508
- 2024LLaVA-CoT: Let Vision Language Models Reason Step-by-Stepcited by 446
- 2023RemoteCLIP: A Vision Language Foundation Model for Remote Sensingcited by 357
- 2024Molmo and PixMo: Open Weights and Open Data for State-of-the-Art Vision-Language Modelscited by 321
- 2024VisionZip: Longer is Better but Not Necessary in Vision Language Modelscited by 298
20 more
- 2024Video-XL: Extra-Long Vision Language Model for Hour-Scale Video Understandingcited by 210
- 2020Drug discovery with explainable artificial intelligencecited by 118
- 2024SkyEyeGPT: Unifying Remote Sensing Vision-Language Tasks via Instruction Tuning with Large Language Modelcited by 112
- 2025Words or Vision: Do Vision-Language Models Have Blind Faith in Text?cited by 86
- 2024FastVLM: Efficient Vision Encoding for Vision Language Modelscited by 76
- 2025Med3DVLM: An Efficient Vision-Language Model for 3D Medical Image Analysiscited by 44
- 2025VLM4D: Towards Spatiotemporal Awareness in Vision Language Modelscited by 43
- 2019Explainable artificial intelligence model to predict acute critical illness from electronic health recordscited by 42
- 2025Optimizing the Optimizer for Physics-Informed Neural Networks and Kolmogorov-Arnold Networkscited by 41
- 2023Multi-modal Machine Learning in Engineering Design: A Review and Future Directionscited by 27
- 2024LHRS-Bot-Nova: Improved Multimodal Large Language Model for Remote Sensing Vision-Language Interpretationcited by 25
- 2024FC-KAN: Function Combinations in Kolmogorov-Arnold Networkscited by 18
- 2024Data-driven model discovery with Kolmogorov-Arnold networkscited by 9
- 2024Multifidelity Kolmogorov-Arnold Networkscited by 9
- 2024HumanVLM: Foundation for Human-Scene Vision-Language Modelcited by 8
- 2024Exploring Kolmogorov-Arnold Networks for Interpretable Time Series Classificationcited by 8
- 2025Open-Source Autonomous Driving Software Platforms: Comparison of Autoware and Apollocited by 6
- 2025A Design Framework for operationalizing Trustworthy Artificial Intelligence in Healthcare: Requirements, Tradeoffs and Challenges for its Clinical Adoptioncited by 3
- 2022Vehicle and License Plate Recognition with Novel Dataset for Toll Collectioncited by 2
- 2024Smart Parking with Pixel-Wise ROI Selection for Vehicle Detection Using YOLOv8, YOLOv9, YOLOv10, and YOLOv11cited by 2
Built on by
Not cited yet.
Similar, not cited
Close in wording, no citation either way
- 2024Detecting the Undetectable: Combining Kolmogorov-Arnold Networks and MLP for AI-Generated Image Detection
- 2025Roboflow100-VL: A Multi-Domain Object Detection Benchmark for Vision-Language Models
- 2024Advancing Object Detection in Transportation with Multimodal Large Language Models (MLLMs): A Comprehensive Review and Empirical Testing
- 2024YOLO-World: Real-Time Open-Vocabulary Object Detection
- 2025All You Need for Object Detection: From Pixels, Points, and Prompts to Next-Gen Fusion and Multimodal LLMs/VLMs in Autonomous Vehicles
- 2025Robust Object Detection with Pseudo Labels from VLMs using Per-Object Co-teaching
- 2024Integrating Object Detection Modality into Visual Language Model for Enhanced Autonomous Driving Agent
- 2026Cross-Paradigm Evaluation of Gaze-Based Semantic Object Identification for Intelligent Vehicles