# YOLO with Kolmogorov-Arnold networks and vision-language foundation models for interpretable object detection with trustworthy multimodal AI in computer vision perception

arXiv: https://arxiv.org/abs/2603.23037 · HTML: https://mlsanity.com/paper/2603.23037/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Placed in Driving VLMs and open-vocabulary detection, one of 16 directions in Multimodal LLMs / vision-language models.
- Builds on 32 of its 115 references that are in the map.
- First posted on arXiv 24 Mar 2026.

## Placed in

- Multimodal LLMs / vision-language models / [Driving VLMs and open-vocabulary detection](https://mlsanity.com/mllm/driving-vlms-and-open-vocabulary-detection/)

## Builds on (most cited first, 30 of 32)

- [The Llama 3 Herd of Models](https://mlsanity.com/paper/2407.21783/) (2024)
- [You Only Look Once: Unified, Real-Time Object Detection](https://mlsanity.com/paper/1506.02640/) (2015)
- [InternVL: Scaling up Vision Foundation Models and Aligning for Generic Visual-Linguistic Tasks](https://mlsanity.com/paper/2312.14238/) (2023)
- [Learning to Prompt for Vision-Language Models](https://mlsanity.com/paper/2109.01134/) (2021)
- [YOLOX: Exceeding YOLO Series in 2021](https://mlsanity.com/paper/2107.08430/) (2021)
- [EfficientDet: Scalable and Efficient Object Detection](https://mlsanity.com/paper/1911.09070/) (2019)
- [Object Detection with Deep Learning: A Review](https://mlsanity.com/paper/1807.05511/) (2018)
- [YOLOv10: Real-Time End-to-End Object Detection](https://mlsanity.com/paper/2405.14458/) (2024)
- [LLaVA-CoT: Let Vision Language Models Reason Step-by-Step](https://mlsanity.com/paper/2411.10440/) (2024)
- [RemoteCLIP: A Vision Language Foundation Model for Remote Sensing](https://mlsanity.com/paper/2306.11029/) (2023)
- [Molmo and PixMo: Open Weights and Open Data for State-of-the-Art Vision-Language Models](https://mlsanity.com/paper/2409.17146/) (2024)
- [VisionZip: Longer is Better but Not Necessary in Vision Language Models](https://mlsanity.com/paper/2412.04467/) (2024)
- [Video-XL: Extra-Long Vision Language Model for Hour-Scale Video Understanding](https://mlsanity.com/paper/2409.14485/) (2024)
- [Drug discovery with explainable artificial intelligence](https://mlsanity.com/paper/2007.00523/) (2020)
- [SkyEyeGPT: Unifying Remote Sensing Vision-Language Tasks via Instruction Tuning with Large Language Model](https://mlsanity.com/paper/2401.09712/) (2024)
- [Words or Vision: Do Vision-Language Models Have Blind Faith in Text?](https://mlsanity.com/paper/2503.02199/) (2025)
- [FastVLM: Efficient Vision Encoding for Vision Language Models](https://mlsanity.com/paper/2412.13303/) (2024)
- [Med3DVLM: An Efficient Vision-Language Model for 3D Medical Image Analysis](https://mlsanity.com/paper/2503.20047/) (2025)
- [VLM4D: Towards Spatiotemporal Awareness in Vision Language Models](https://mlsanity.com/paper/2508.02095/) (2025)
- [Explainable artificial intelligence model to predict acute critical illness from electronic health records](https://mlsanity.com/paper/1912.01266/) (2019)
- [Optimizing the Optimizer for Physics-Informed Neural Networks and Kolmogorov-Arnold Networks](https://mlsanity.com/paper/2501.16371/) (2025)
- [Multi-modal Machine Learning in Engineering Design: A Review and Future Directions](https://mlsanity.com/paper/2302.10909/) (2023)
- [LHRS-Bot-Nova: Improved Multimodal Large Language Model for Remote Sensing Vision-Language Interpretation](https://mlsanity.com/paper/2411.09301/) (2024)
- [FC-KAN: Function Combinations in Kolmogorov-Arnold Networks](https://mlsanity.com/paper/2409.01763/) (2024)
- [Data-driven model discovery with Kolmogorov-Arnold networks](https://mlsanity.com/paper/2409.15167/) (2024)
- [Multifidelity Kolmogorov-Arnold Networks](https://mlsanity.com/paper/2410.14764/) (2024)
- [HumanVLM: Foundation for Human-Scene Vision-Language Model](https://mlsanity.com/paper/2411.03034/) (2024)
- [Exploring Kolmogorov-Arnold Networks for Interpretable Time Series Classification](https://mlsanity.com/paper/2411.14904/) (2024)
- [Open-Source Autonomous Driving Software Platforms: Comparison of Autoware and Apollo](https://mlsanity.com/paper/2501.18942/) (2025)
- [A Design Framework for operationalizing Trustworthy Artificial Intelligence in Healthcare: Requirements, Tradeoffs and Challenges for its Clinical Adoption](https://mlsanity.com/paper/2504.19179/) (2025)

## Abstract (arXiv)

The trustworthy object detection capabilities of a novel Kolmogorov-Arnold network framework are examined here. The approach addresses a key limitation in computer vision for vehicle detection perception, and beyond. These systems offer limited transparency regarding the reliability of their confidence scores in visually degraded or ambiguous scenes. To this end, a Kolmogorov-Arnold network is employed as an interpretable post-hoc surrogate to model the trustworthiness of the You Only Look Once (Yolov10) detections using seven geometric and semantic features. The additive spline-based structure of the Kolmogorov-Arnold network enables direct visualisation of each feature's influence. This produces smooth and transparent functional mappings that reveal when the model's confidence is well supported and when it is unreliable. Furthermore, a bootstrapped language-image (BLIP) foundation model generates descriptive captions of each scene. This tool enables a lightweight multimodal interface without affecting the interpretability layer. Experiments on both Common Objects in Context (COCO), and images from the University of Bath campus demonstrate that the framework accurately identifies low-trust predictions under blur, occlusion, or low texture. This provides actionable insights for acceptance, review, or downstream risk mitigation. The resulting system delivers interpretable object detection with trustworthy confidence estimates. It offers a powerful tool for transparent and practical perception component for autonomous and multimodal artificial intelligence applications.