# Effective Receptive Field Ordering Matters for Infrared Small Target Detection

arXiv: https://arxiv.org/abs/2607.23994 · HTML: https://mlsanity.com/paper/2607.23994/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 39 of its 58 references that are in the map.
- First posted on arXiv 27 Jul 2026.

## Builds on (most cited first, 30 of 39)

- [Transformers are SSMs: Generalized Models and Efficient Algorithms Through Structured State Space Duality](https://mlsanity.com/paper/2405.21060/) (2024)
- [Visualizing the Loss Landscape of Neural Nets](https://mlsanity.com/paper/1712.09913/) (2017)
- [Understanding the Effective Receptive Field in Deep Convolutional Neural Networks](https://mlsanity.com/paper/1701.04128/) (2017)
- [Dynamic Neural Networks: A Survey](https://mlsanity.com/paper/2102.04906/) (2021)
- [MS-TCN: Multi-Stage Temporal Convolutional Network for Action Segmentation](https://mlsanity.com/paper/1903.01945/) (2019)
- [U-KAN Makes Strong Backbone for Medical Image Segmentation and Generation](https://mlsanity.com/paper/2406.02918/) (2024)
- [Dense Nested Attention Network for Infrared Small Target Detection](https://mlsanity.com/paper/2106.00487/) (2021)
- [UIU-Net: U-Net in U-Net for Infrared Small Object Detection](https://mlsanity.com/paper/2212.00968/) (2022)
- [Attentional Local Contrast Networks for Infrared Small Target Detection](https://mlsanity.com/paper/2012.08573/) (2020)
- [Conv2Former: A Simple Transformer-Style ConvNet for Visual Recognition](https://mlsanity.com/paper/2211.11943/) (2022)
- [Infrared Small Target Detection with Scale and Location Sensitivity](https://mlsanity.com/paper/2403.19366/) (2024)
- [Efficient Deformable ConvNets: Rethinking Dynamic and Sparse Operator for Vision Applications](https://mlsanity.com/paper/2401.06197/) (2024)
- [Frequency-aware Feature Fusion for Dense Image Prediction](https://mlsanity.com/paper/2408.12879/) (2024)
- [IRSAM: Advancing Segment Anything Model for Infrared Small Target Detection](https://mlsanity.com/paper/2407.07520/) (2024)
- [Adaptive Frequency Filters As Efficient Global Token Mixers](https://mlsanity.com/paper/2307.14008/) (2023)
- [MB-TaylorFormer V2: Improved Multi-branch Linear Transformer Expanded by Taylor Formula for Image Restoration](https://mlsanity.com/paper/2501.04486/) (2025)
- [Pinwheel-shaped Convolution and Scale-based Dynamic Loss for Infrared Small Target Detection](https://mlsanity.com/paper/2412.16986/) (2024)
- [YOLO-MS: Rethinking Multi-Scale Representation Learning for Real-time Object Detection](https://mlsanity.com/paper/2308.05480/) (2023)
- [A Simple yet Effective Network based on Vision Transformer for Camouflaged Object and Salient Object Detection](https://mlsanity.com/paper/2402.18922/) (2024)
- [Frequency Dynamic Convolution for Dense Image Prediction](https://mlsanity.com/paper/2503.18783/) (2025)
- [Towards Understanding the Importance of Shortcut Connections in Residual Networks](https://mlsanity.com/paper/1909.04653/) (2019)
- [Pick of the Bunch: Detecting Infrared Small Targets Beyond Hit-Miss Trade-Offs via Selective Rank-Aware Attention](https://mlsanity.com/paper/2408.03717/) (2024)
- [Low-Resolution Self-Attention for Semantic Segmentation](https://mlsanity.com/paper/2310.05026/) (2023)
- [RPCANet++: Deep Interpretable Robust PCA for Sparse Object Segmentation](https://mlsanity.com/paper/2508.04190/) (2025)
- [On the Convergence of Learning-based Iterative Methods for Nonconvex Inverse Problems](https://mlsanity.com/paper/1808.05331/) (2018)
- [Probing Deep into Temporal Profile Makes the Infrared Small Target Detector Much Better](https://mlsanity.com/paper/2506.12766/) (2025)
- [RFAConv: Receptive-Field Attention Convolution for Improving Convolutional Neural Networks](https://mlsanity.com/paper/2304.03198/) (2023)
- [DRPCA-Net: Make Robust PCA Great Again for Infrared Small Target Detection](https://mlsanity.com/paper/2507.09541/) (2025)
- [Scaling Up Your Kernels: Large Kernel Design in ConvNets towards Universal Representations](https://mlsanity.com/paper/2410.08049/) (2024)
- [Seeing Through the Noise: Improving Infrared Small Target Detection and Segmentation from Noise Suppression Perspective](https://mlsanity.com/paper/2508.06878/) (2025)

## Abstract (arXiv)

In this work, we investigate a previously unexplored architectural dimension for infrared small target detection: the organization of effective receptive fields (ERFs) during feature refinement. Unlike existing approaches that primarily improve individual feature operators, we argue that ERF organization constitutes an architectural dimension independent of receptive field design itself, and formulate deep feature transformation as a progressive residual correction process, from which a theoretical framework for ERF scheduling is established. Specifically, we reveal that ERF refinement is governed by two fundamental properties: scale-frequency correspondence, which aligns different ERF scales with distinct residual frequency characteristics, and nonlinear non-commutativity, which makes different ERF orderings produce fundamentally different refinement trajectories. Together, these properties show that ERF organization, rather than ERF scale alone, governs refinement dynamics. Guided by these principles, we propose Receptive Field Ordering Network (RFONet), which realizes hierarchical ERF scheduling through a multigrid-inspired V-cycle strategy using only standard 3×3 convolutions. RFONet achieves state-of-the-art performance on multiple benchmarks with only 1.16M parameters and over 157 FPS inference speed. Beyond empirical performance, our theoretical analysis provides theoretical guarantees for stable residual refinement under perturbations, frequency shifts, and partial occlusions, which are consistently reflected in superior noise robustness and cross-dataset generalization. Finally, our framework reformulates ERF organization as a task-dependent optimization objective, providing a principled foundation for future adaptive receptive field scheduling.