# Transferability of Convolutional Neural Networks in Stationary Learning Tasks

arXiv: https://arxiv.org/abs/2307.11588 · HTML: https://mlsanity.com/paper/2307.11588/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 29 of its 62 references that are in the map.
- First posted on arXiv 21 Jul 2023.

## Builds on (most cited first, 29 of 29)

- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [High-Resolution Image Synthesis with Latent Diffusion Models](https://mlsanity.com/paper/2112.10752/) (2021)
- [LLaMA: Open and Efficient Foundation Language Models](https://mlsanity.com/paper/2302.13971/) (2023)
- [Taming Transformers for High-Resolution Image Synthesis](https://mlsanity.com/paper/2012.09841/) (2020)
- [Optuna: A Next-generation Hyperparameter Optimization Framework](https://mlsanity.com/paper/1907.10902/) (2019)
- [4D Spatio-Temporal ConvNets: Minkowski Convolutional Neural Networks](https://mlsanity.com/paper/1904.08755/) (2019)
- [Scaling Language Models: Methods, Analysis & Insights from Training Gopher](https://mlsanity.com/paper/2112.11446/) (2021)
- [A Comprehensive Survey on Transfer Learning](https://mlsanity.com/paper/1911.02685/) (2019)
- [Carbon Emissions and Large Neural Network Training](https://mlsanity.com/paper/2104.10350/) (2021)
- [Using DeepSpeed and Megatron to Train Megatron-Turing NLG 530B, A Large-Scale Generative Language Model](https://mlsanity.com/paper/2201.11990/) (2022)
- [A Survey on Deep Transfer Learning](https://mlsanity.com/paper/1808.01974/) (2018)
- [A Survey of Transformers](https://mlsanity.com/paper/2106.04554/) (2021)
- [A Survey of Convolutional Neural Networks: Analysis, Applications, and Prospects](https://mlsanity.com/paper/2004.02806/) (2020)
- [Recent Advances in Convolutional Neural Networks](https://mlsanity.com/paper/1512.07108/) (2015)
- [Transfer Learning in Deep Reinforcement Learning: A Survey](https://mlsanity.com/paper/2009.07888/) (2020)
- [Submanifold Sparse Convolutional Networks](https://mlsanity.com/paper/1706.01307/) (2017)
- [Deep Convolutional Neural Networks for Computer-Aided Detection: CNN Architectures, Dataset Characteristics and Transfer Learning](https://mlsanity.com/paper/1602.03409/) (2016)
- [The Computational Limits of Deep Learning](https://mlsanity.com/paper/2007.05558/) (2020)
- [Graphon Neural Networks and the Transferability of Graph Neural Networks](https://mlsanity.com/paper/2006.03548/) (2020)
- [On Lipschitz Bounds of General Convolutional Neural Networks](https://mlsanity.com/paper/1808.01415/) (2018)
- [Graph Neural Networks for Decentralized Controllers](https://mlsanity.com/paper/2003.10280/) (2020)
- [Graph Neural Networks for Multi-Robot Active Information Acquisition](https://mlsanity.com/paper/2209.12091/) (2022)
- [Mobile Wireless Network Infrastructure on Demand](https://mlsanity.com/paper/2002.03026/) (2020)
- [Derivation of a Constant Velocity Motion Model for Visual Tracking](https://mlsanity.com/paper/2005.00844/) (2020)
- [Multi-sensor Joint Adaptive Birth Sampler for Labeled Random Finite Set Tracking](https://mlsanity.com/paper/2109.04355/) (2021)
- [CNNs Avoid Curse of Dimensionality by Learning on Patches](https://mlsanity.com/paper/2205.10760/) (2022)
- [Multi-Target Tracking with Transferable Convolutional Neural Networks](https://mlsanity.com/paper/2210.15539/) (2022)
- [Solving Large-scale Spatial Problems with Convolutional Neural Networks](https://mlsanity.com/paper/2306.08191/) (2023)
- [Learning Connectivity-Maximizing Network Configurations](https://mlsanity.com/paper/2112.07663/) (2021)

## Built on by (most cited first, 1 of 1)

- [Limit Analysis of Graph Neural Networks with Wireless Conflict Graphs](https://mlsanity.com/paper/2606.03794/) (2026)

## Abstract (arXiv)

Recent advances in hardware and big data acquisition have accelerated the development of deep learning techniques. For an extended period of time, increasing the model complexity has led to performance improvements for various tasks. However, this trend is becoming unsustainable and there is a need for alternative, computationally lighter methods. In this paper, we introduce a novel framework for efficient training of convolutional neural networks (CNNs) for large-scale spatial problems. To accomplish this we investigate the properties of CNNs for tasks where the underlying signals are stationary. We show that a CNN trained on small windows of such signals achieves a nearly performance on much larger windows without retraining. This claim is supported by our theoretical analysis, which provides a bound on the performance degradation. Additionally, we conduct thorough experimental analysis on two tasks: multi-target tracking and mobile infrastructure on demand. Our results show that the CNN is able to tackle problems with many hundreds of agents after being trained with fewer than ten. Thus, CNN architectures provide solutions to these problems at previously computationally intractable scales.