# Pick of the Bunch: Detecting Infrared Small Targets Beyond Hit-Miss Trade-Offs via Selective Rank-Aware Attention

arXiv: https://arxiv.org/abs/2408.03717 · HTML: https://mlsanity.com/paper/2408.03717/

## Key facts

- Cited by 17 papers in the map (data through 14 Oct 2026).
- Builds on 32 of its 72 references that are in the map.
- First posted on arXiv 7 Aug 2024.

## Builds on (most cited first, 30 of 32)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [U-Net: Convolutional Networks for Biomedical Image Segmentation](https://mlsanity.com/paper/1505.04597/) (2015)
- [Squeeze-and-Excitation Networks](https://mlsanity.com/paper/1709.01507/) (2017)
- [Pyramid Vision Transformer: A Versatile Backbone for Dense Prediction without Convolutions](https://mlsanity.com/paper/2102.12122/) (2021)
- [UNet++: A Nested U-Net Architecture for Medical Image Segmentation](https://mlsanity.com/paper/1807.10165/) (2018)
- [Attention U-Net: Learning Where to Look for the Pancreas](https://mlsanity.com/paper/1804.03999/) (2018)
- [CCNet: Criss-Cross Attention for Semantic Segmentation](https://mlsanity.com/paper/1811.11721/) (2018)
- [Selective Kernel Networks](https://mlsanity.com/paper/1903.06586/) (2019)
- [Road Extraction by Deep Residual U-Net](https://mlsanity.com/paper/1711.10684/) (2017)
- [UNet 3+: A Full-Scale Connected UNet for Medical Image Segmentation](https://mlsanity.com/paper/2004.08790/) (2020)
- [Learning A Sparse Transformer Network for Effective Image Deraining](https://mlsanity.com/paper/2303.11950/) (2023)
- [MultiResUNet : Rethinking the U-Net Architecture for Multimodal Biomedical Image Segmentation](https://mlsanity.com/paper/1902.04049/) (2019)
- [Gather-Excite: Exploiting Feature Context in Convolutional Neural Networks](https://mlsanity.com/paper/1810.12348/) (2018)
- [Asymmetric Contextual Modulation for Infrared Small Target Detection](https://mlsanity.com/paper/2009.14530/) (2020)
- [Dense Nested Attention Network for Infrared Small Target Detection](https://mlsanity.com/paper/2106.00487/) (2021)
- [QuadTree Attention for Vision Transformers](https://mlsanity.com/paper/2201.02767/) (2022)
- [UIU-Net: U-Net in U-Net for Infrared Small Object Detection](https://mlsanity.com/paper/2212.00968/) (2022)
- [Attentional Feature Fusion](https://mlsanity.com/paper/2009.14082/) (2020)
- [Attentional Local Contrast Networks for Infrared Small Target Detection](https://mlsanity.com/paper/2012.08573/) (2020)
- [NAS-FAS: Static-Dynamic Central Difference Network Search for Face Anti-Spoofing](https://mlsanity.com/paper/2011.02062/) (2020)
- [Large Selective Kernel Network for Remote Sensing Object Detection](https://mlsanity.com/paper/2303.09030/) (2023)
- [DUNet: A deformable network for retinal vessel segmentation](https://mlsanity.com/paper/1811.01206/) (2018)
- [Bi-Directional ConvLSTM U-Net with Densley Connected Convolutions](https://mlsanity.com/paper/1909.00166/) (2019)
- [Reweighted Infrared Patch-Tensor Model With Both Non-Local and Local Priors for Single-Frame Small Target Detection](https://mlsanity.com/paper/1703.09157/) (2017)
- [Memory-efficient Transformers via Top-k Attention](https://mlsanity.com/paper/2106.06899/) (2021)
- [MTU-Net: Multi-level TransUNet for Space-based Infrared Tiny Ship Detection](https://mlsanity.com/paper/2209.13756/) (2022)
- [LSKNet: A Foundation Lightweight Backbone for Remote Sensing](https://mlsanity.com/paper/2403.11735/) (2024)
- [BiO-Net: Learning Recurrent Bi-directional Connections for Encoder-Decoder Architecture](https://mlsanity.com/paper/2007.00243/) (2020)
- [One-Stage Cascade Refinement Networks for Infrared Small Target Detection](https://mlsanity.com/paper/2212.08472/) (2022)

## Built on by (most cited first, 17 of 17)

- [SM3Det: A Unified Model for Multi-Modal Remote Sensing Object Detection](https://mlsanity.com/paper/2412.20665/) (2024)
- [Deep learning based infrared small object segmentation: Challenges and future directions](https://mlsanity.com/paper/2502.14168/) (2025)
- [Rethinking Evaluation of Infrared Small Target Detection](https://mlsanity.com/paper/2509.16888/) (2025)
- [A Simple Detector with Frame Dynamics is a Strong Tracker](https://mlsanity.com/paper/2505.04917/) (2025)
- [SeqCSIST: Sequential Closely-Spaced Infrared Small Target Unmixing](https://mlsanity.com/paper/2507.09556/) (2025)
- [It's Not the Target, It's the Background: Rethinking Infrared Small Target Detection via Deep Patch-Free Low-Rank Representations](https://mlsanity.com/paper/2506.10425/) (2025)
- [Unifying Heterogeneous Multi-Modal Remote Sensing Detection Via Language-Pivoted Pretraining](https://mlsanity.com/paper/2603.01758/) (2026)
- [DSCSNet: A Dynamic Sparse Compression Sensing Network for Closely-Spaced Infrared Small Target Unmixing](https://mlsanity.com/paper/2603.21192/) (2026)
- [FSGNet: A Frequency-Aware and Semantic Guidance Network for Infrared Small Target Detection](https://mlsanity.com/paper/2603.25389/) (2026)
- [LoHGNet: Infrared Small Target Detection through Lorentz Geometric Encoding with High-Order Relation Learning](https://mlsanity.com/paper/2605.07213/) (2026)
- [Learning Dynamic Local Context Representations for Infrared Small Target Detection](https://mlsanity.com/paper/2412.17401/) (2024)
- [HyperTea: A Hypergraph-based Temporal Enhancement and Alignment Network for Moving Infrared Small Target Detection](https://mlsanity.com/paper/2508.10678/) (2025)
- [STGBD-Net: Spatio-temporal Gradient Basis Decomposition Network for Infrared Small Target Detection](https://mlsanity.com/paper/2512.03470/) (2025)
- [DCCS-Det: Directional Context and Cross-Scale-Aware Detector for Infrared Small Target](https://mlsanity.com/paper/2601.16428/) (2026)
- [Rethinking Representations for Cross-Domain Infrared Small Target Detection: A Generalizable Perspective from the Frequency Domain](https://mlsanity.com/paper/2604.01934/) (2026)
- [Effective Receptive Field Ordering Matters for Infrared Small Target Detection](https://mlsanity.com/paper/2607.23994/) (2026)
- [RDANet: Relative Degradation Aware Network for Infrared Small Target Detection](https://mlsanity.com/paper/2608.20870/) (2026)

## Abstract (arXiv)

Infrared small target detection faces the inherent challenge of precisely localizing dim targets amidst complex background clutter. Traditional approaches struggle to balance detection precision and false alarm rates. To break this dilemma, we propose SeRankDet, a deep network that achieves high accuracy beyond the conventional hit-miss trade-off, by following the ``Pick of the Bunch'' principle. At its core lies our Selective Rank-Aware Attention (SeRank) module, employing a non-linear Top-K selection process that preserves the most salient responses, preventing target signal dilution while maintaining constant complexity. Furthermore, we replace the static concatenation typical in U-Net structures with our Large Selective Feature Fusion (LSFF) module, a dynamic fusion strategy that empowers SeRankDet with adaptive feature integration, enhancing its ability to discriminate true targets from false alarms. The network's discernment is further refined by our Dilated Difference Convolution (DDC) module, which merges differential convolution aimed at amplifying subtle target characteristics with dilated convolution to expand the receptive field, thereby substantially improving target-background separation. Despite its lightweight architecture, the proposed SeRankDet sets new benchmarks in state-of-the-art performance across multiple public datasets. The code is available at https://github.com/GrokCV/SeRankDet.