Understand Before Detect: Vision--Language Learning for Omni-Domain Infrared Small Target Detection
Not on the map. Why?
Abstract (arXiv)
Omni-domain infrared small target (IRST) detection is crucial for infrared surveillance, yet remains challenging due to heterogeneous imaging domains and inconsistent target characteristics. Previous deep learning-based methods have been developed for visual-only paradigms and achieved promising performance on domain-specific tasks. However, existing methods follow the task-specific supervised learning paradigm. This paradigm simplifies the full-scene infrared observations to sparse target supervision, discarding the semantics that remain invariant across heterogeneous domains. Consequently, detection performance suffers substantially under domain shifts. To handle this issue, we introduce ``understand before detect'', a paradigm that formulates omni-domain IRST detection as an understanding-driven process, where holistic infrared target understanding precedes precise detection. Building on this paradigm, we propose JinSight, which first develops holistic IRST understanding through language supervision and then transfers the learned cross-domain representations to precise small-target detection. By grounding infrared representations in language semantics, JinSight enables a single model to generalize across heterogeneous infrared domains. We then introduce Latent Semantic Interaction (LSI), which exchanges language-aligned global semantics with fine-grained spatial features in a compact low-rank space. To address the lack of multimodal omni-domain IRST benchmarks, we build OmniIRST-VL, the first large-scale, highly diverse vision--language dataset for omni-domain IRST detection. It comprises over 39k annotations across six complementary instruction tasks covering both scene-level understanding and target-centric reasoning.
Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 22 of its 33 references that are in the map. First posted on arXiv 7 Aug 2026.
Builds on
22 of 33 references.
References · 22
- 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
- 2015U-Net: Convolutional Networks for Biomedical Image Segmentationcited by 19,604
- 2016Feature Pyramid Networks for Object Detectioncited by 5,021
- 2023Improved Baselines with Visual Instruction Tuningcited by 4,362
- 2021SegFormer: Simple and Efficient Design for Semantic Segmentation with Transformerscited by 2,923
- 2023InternVL: Scaling up Vision Foundation Models and Aligning for Generic Visual-Linguistic Taskscited by 2,534
- 2025Qwen3-VL Technical Reportcited by 2,534
- 2014CIDEr: Consensus-based Image Description Evaluationcited by 2,027
- 2018Unified Perceptual Parsing for Scene Understandingcited by 1,265
- 2022When and why vision-language models behave like bags-of-words, and what to do about it?cited by 603
- 2022Vision Transformer Adapter for Dense Predictionscited by 564
- 2019ResUNet-a: a deep learning framework for semantic segmentation of remotely sensed datacited by 260
10 more
- 2023SkySense: A Multi-Modal Remote Sensing Foundation Model Towards Universal Interpretation for Earth Observation Imagerycited by 202
- 2024EarthGPT: A Universal Multi-modal Large Language Model for Multi-sensor Image Comprehension in Remote Sensing Domaincited by 181
- 2024LHRS-Bot: Empowering Remote Sensing with VGI-Enhanced Large Multimodal Language Modelcited by 133
- 2024EarthDial: Turning Multi-sensory Earth Observations to Interactive Dialoguescited by 81
- 2024Infrared Small Target Detection with Scale and Location Sensitivitycited by 70
- 2022MTU-Net: Multi-level TransUNet for Space-based Infrared Tiny Ship Detectioncited by 47
- 2025Probing Deep into Temporal Profile Makes the Infrared Small Target Detector Much Bettercited by 11
- 2025Text-IRSTD: Leveraging Semantic Text to Promote Infrared Small Target Detection in Complex Scenescited by 7
- 2025Seeing Through the Noise: Improving Infrared Small Target Detection and Segmentation from Noise Suppression Perspectivecited by 7
- 2025Leveraging Language Prior for Infrared Small Target Detectioncited by 1
Built on by
Not cited yet.