# Size-invariance Matters: Rethinking Metrics and Losses for Imbalanced Multi-object Salient Object Detection

arXiv: https://arxiv.org/abs/2405.09782 · HTML: https://mlsanity.com/paper/2405.09782/

## Key facts

- Cited by 14 papers in the map (data through 14 Oct 2026).
- Builds on 32 of its 73 references that are in the map.
- First posted on arXiv 16 May 2024.

## Builds on (most cited first, 30 of 32)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [U-Net: Convolutional Networks for Biomedical Image Segmentation](https://mlsanity.com/paper/1505.04597/) (2015)
- [Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks](https://mlsanity.com/paper/1506.01497/) (2015)
- [You Only Look Once: Unified, Real-Time Object Detection](https://mlsanity.com/paper/1506.02640/) (2015)
- [V-Net: Fully Convolutional Neural Networks for Volumetric Medical Image Segmentation](https://mlsanity.com/paper/1606.04797/) (2016)
- [Rich feature hierarchies for accurate object detection and semantic segmentation](https://mlsanity.com/paper/1311.2524/) (2013)
- [Enhanced-alignment Measure for Binary Foreground Map Evaluation](https://mlsanity.com/paper/1805.10421/) (2018)
- [Causal Intervention for Weakly-Supervised Semantic Segmentation](https://mlsanity.com/paper/2009.12547/) (2020)
- [A Simple Pooling-Based Design for Real-Time Salient Object Detection](https://mlsanity.com/paper/1904.09569/) (2019)
- [UnitBox: An Advanced Object Detection Network](https://mlsanity.com/paper/1608.01471/) (2016)
- [Object Detection in Aerial Images: A Large-Scale Benchmark and Challenges](https://mlsanity.com/paper/2102.12219/) (2021)
- [Visual Saliency Based on Multiscale Deep Features](https://mlsanity.com/paper/1503.08663/) (2015)
- [What do different evaluation metrics tell us about saliency models?](https://mlsanity.com/paper/1604.03605/) (2016)
- [Salient Object Detection in the Deep Learning Era: An In-Depth Survey](https://mlsanity.com/paper/1904.09146/) (2019)
- [A vector-contraction inequality for Rademacher complexities](https://mlsanity.com/paper/1605.00251/) (2016)
- [Few-Shot Object Detection and Viewpoint Estimation for Objects in the Wild](https://mlsanity.com/paper/2007.12107/) (2020)
- [Salient Object Detection via Integrity Learning](https://mlsanity.com/paper/2101.07663/) (2021)
- [Label Decoupling Framework for Salient Object Detection](https://mlsanity.com/paper/2008.11048/) (2020)
- [Salient Object Detection: A Survey](https://mlsanity.com/paper/1411.5878/) (2014)
- [Amulet: Aggregating Multi-level Convolutional Features for Salient Object Detection](https://mlsanity.com/paper/1708.02001/) (2017)
- [EDN: Salient Object Detection via Extremely-Downsampled Network](https://mlsanity.com/paper/2012.13093/) (2020)
- [Learning Generative Vision Transformer with Energy-Based Latent Space for Saliency Prediction](https://mlsanity.com/paper/2112.13528/) (2021)
- [Deep Contrast Learning for Salient Object Detection](https://mlsanity.com/paper/1603.01976/) (2016)
- [Accurate RGB-D Salient Object Detection via Collaborative Learning](https://mlsanity.com/paper/2007.11782/) (2020)
- [Learning Uncertain Convolutional Features for Accurate Saliency Detection](https://mlsanity.com/paper/1708.02031/) (2017)
- [A General Framework for Consistent Structured Prediction with Implicit Loss Embeddings](https://mlsanity.com/paper/2002.05424/) (2020)
- [PiCANet: Pixel-wise Contextual Attention Learning for Accurate Saliency Detection](https://mlsanity.com/paper/1812.06314/) (2018)
- [Regularized Densely-connected Pyramid Network for Salient Instance Segmentation](https://mlsanity.com/paper/2008.12416/) (2020)
- [Optimizing Two-way Partial AUC with an End-to-end Framework](https://mlsanity.com/paper/2206.11655/) (2022)
- [CNN-based RGB-D Salient Object Detection: Learn, Select and Fuse](https://mlsanity.com/paper/1909.09309/) (2019)

## Built on by (most cited first, 14 of 14)

- [AUCSeg: AUC-oriented Pixel-level Long-tail Semantic Segmentation](https://mlsanity.com/paper/2409.20398/) (2024)
- [One Image is Worth a Thousand Words: A Usability Preservable Text-Image Collaborative Erasing Framework](https://mlsanity.com/paper/2505.11131/) (2025)
- [Balancing the Scales: A Theoretical and Algorithmic Framework for Learning from Imbalanced Data](https://mlsanity.com/paper/2502.10381/) (2025)
- [Improved Balanced Classification with Theoretically Grounded Loss Functions](https://mlsanity.com/paper/2512.23947/) (2025)
- [Adaptive Event Stream Slicing for Open-Vocabulary Event-Based Object Detection via Vision-Language Knowledge Distillation](https://mlsanity.com/paper/2510.00681/) (2025)
- [Rethinking Detecting Salient and Camouflaged Objects in Unconstrained Scenes](https://mlsanity.com/paper/2412.10943/) (2024)
- [Dual-Stage Reweighted MoE for Long-Tailed Egocentric Mistake Detection](https://mlsanity.com/paper/2509.12990/) (2025)
- [From Static to Dynamic: Exploring Self-supervised Image-to-Video Representation Transfer Learning](https://mlsanity.com/paper/2603.26597/) (2026)
- [Optimized Deferral for Imbalanced Settings](https://mlsanity.com/paper/2604.27723/) (2026)
- [Divide, Weight, and Route: Difficulty-Aware Optimization with Dynamic Expert Fusion for Long-tailed Recognition](https://mlsanity.com/paper/2508.19630/) (2025)
- [Deconstructing the Failure of Ideal Noise Correction: A Three-Pillar Diagnosis](https://mlsanity.com/paper/2603.12997/) (2026)
- [Beyond Pixel Overlap: A Framework for Decomposing Segmentation Evaluation Metrics](https://mlsanity.com/paper/2607.00886/) (2026)
- [Towards Size-invariant Salient Object Detection: A Generic Evaluation and Optimization Approach](https://mlsanity.com/paper/2509.15573/) (2025)
- [Rethinking Conditional Generation for Underwater Salient Object Detection](https://mlsanity.com/paper/2607.01825/) (2026)

## Abstract (arXiv)

This paper explores the size-invariance of evaluation metrics in Salient Object Detection (SOD), especially when multiple targets of diverse sizes co-exist in the same image. We observe that current metrics are size-sensitive, where larger objects are focused, and smaller ones tend to be ignored. We argue that the evaluation should be size-invariant because bias based on size is unjustified without additional semantic information. In pursuit of this, we propose a generic approach that evaluates each salient object separately and then combines the results, effectively alleviating the imbalance. We further develop an optimization framework tailored to this goal, achieving considerable improvements in detecting objects of different sizes. Theoretically, we provide evidence supporting the validity of our new metrics and present the generalization analysis of SOD. Extensive experiments demonstrate the effectiveness of our method. The code is available at https://github.com/Ferry-Li/SI-SOD.