# When Bigger is Worse: A Practitioner's Guide to Model Selection Under Data Scarcity

arXiv: https://arxiv.org/abs/2603.02142 · HTML: https://mlsanity.com/paper/2603.02142/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 22 of its 40 references that are in the map.
- First posted on arXiv 2 Mar 2026.

## Builds on (most cited first, 22 of 22)

- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [Microsoft COCO: Common Objects in Context](https://mlsanity.com/paper/1405.0312/) (2014)
- [Scaling Laws for Neural Language Models](https://mlsanity.com/paper/2001.08361/) (2020)
- [EfficientNet: Rethinking Model Scaling for Convolutional Neural Networks](https://mlsanity.com/paper/1905.11946/) (2019)
- [Feature Pyramid Networks for Object Detection](https://mlsanity.com/paper/1612.03144/) (2016)
- [Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networks](https://mlsanity.com/paper/1703.03400/) (2017)
- [Prototypical Networks for Few-shot Learning](https://mlsanity.com/paper/1703.05175/) (2017)
- [The Lottery Ticket Hypothesis: Finding Sparse, Trainable Neural Networks](https://mlsanity.com/paper/1803.03635/) (2018)
- [How transferable are features in deep neural networks?](https://mlsanity.com/paper/1411.1792/) (2014)
- [Remote Sensing Image Scene Classification: Benchmark and State of the Art](https://mlsanity.com/paper/1703.00121/) (2017)
- [DOTA: A Large-scale Dataset for Object Detection in Aerial Images](https://mlsanity.com/paper/1711.10398/) (2017)
- [xView: Objects in Context in Overhead Imagery](https://mlsanity.com/paper/1802.07856/) (2018)
- [An Analysis of Scale Invariance in Object Detection - SNIP](https://mlsanity.com/paper/1711.08189/) (2017)
- [Modeling Visual Context is Key to Augmenting Object Detection Datasets](https://mlsanity.com/paper/1807.07428/) (2018)
- [Better plain ViT baselines for ImageNet-1k](https://mlsanity.com/paper/2205.01580/) (2022)
- [A Survey on Object Detection in Optical Remote Sensing Images](https://mlsanity.com/paper/1603.06201/) (2016)
- [The Effects of Super-Resolution on Object Detection Performance in Satellite Imagery](https://mlsanity.com/paper/1812.04098/) (2018)
- [A Close Look at Deep Learning with Small Data](https://mlsanity.com/paper/2003.12843/) (2020)
- [BAOD: Budget-Aware Object Detection](https://mlsanity.com/paper/1904.05443/) (2019)
- [A crowdsourced dataset of aerial images with annotated solar photovoltaic arrays and installation metadata](https://mlsanity.com/paper/2209.03726/) (2022)
- [HyperionSolarNet: Solar Panel Detection from Aerial Images](https://mlsanity.com/paper/2201.02107/) (2022)
- [Space-scale Exploration of the Poor Reliability of Deep Learning Models: the Case of the Remote Sensing of Rooftop Photovoltaic Systems](https://mlsanity.com/paper/2408.07828/) (2024)

## Abstract (arXiv)

Scaling laws assume larger models trained on more data consistently outperform smaller ones -- an assumption that drives model selection in computer vision but remains untested in resource-constrained Earth observation (EO). We conduct a systematic efficiency analysis across three scaling dimensions: model size, dataset size, and input resolution, on rooftop photovoltaic (PV) detection in Madagascar, yielding 180 training runs across 60 configurations. Optimizing for model efficiency (mAP50 per unit of model size), we find a consistent efficiency inversion: YOLO11N achieves the highest efficiency (22× higher than YOLO11X) with no accuracy penalty: it reaches the second highest absolute mAP50 (0.459), outperforming all models except YOLO11S by a margin smaller than run-to-run variance, directly contradicting the scaling prior. Resolution is the dominant resource allocation lever: moving from 416 px to 1280 px at 10 % of the data matches the efficiency gain of collecting the full dataset at low resolution. These findings are robust to the deployment objective: small high-resolution configurations are Pareto-dominant across all 60 experimental setups in the joint accuracy-throughput space, leaving no tradeoff to resolve. In data-scarce EO, the scaling prior does not just fail: it inverts.