Your Image is My Video: Reshaping the Receptive Field via Image-To-Video Differentiable AutoAugmentation and Fusion
Not on the map. Why?
Abstract (arXiv)
The landscape of deep learning research is moving towards innovative strategies to harness the true potential of data. Traditionally, emphasis has been on scaling model architectures, resulting in large and complex neural networks, which can be difficult to train with limited computational resources. However, independently of the model size, data quality (i.e. amount and variability) is still a major factor that affects model generalization. In this work, we propose a novel technique to exploit available data through the use of automatic data augmentation for the tasks of image classification and semantic segmentation. We introduce the first Differentiable Augmentation Search method (DAS) to generate variations of images that can be processed as videos. Compared to previous approaches, DAS is extremely fast and flexible, allowing the search on very large search spaces in less than a GPU day. Our intuition is that the increased receptive field in the temporal dimension provided by DAS could lead to benefits also to the spatial receptive field. More specifically, we leverage DAS to guide the reshaping of the spatial receptive field by selecting task-dependant transformations. As a result, compared to standard augmentation alternatives, we improve in terms of accuracy on ImageNet, Cifar10, Cifar100, Tiny-ImageNet, Pascal-VOC-2012 and CityScapes datasets when plugging-in our DAS over different light-weight video backbones.
Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 55 of its 63 references that are in the map. First posted on arXiv 22 Mar 2024.
Builds on
55 of 63 references.
References · 55
- 2015Deep Residual Learning for Image Recognitioncited by 52,096
- 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 30,186
- 2014Very Deep Convolutional Networks for Large-Scale Image Recognitioncited by 15,364
- 2021Swin Transformer: Hierarchical Vision Transformer using Shifted Windowscited by 12,139
- 2023DINOv2: Learning Robust Visual Features without Supervisioncited by 7,974
- 2015Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shiftcited by 7,005
- 2019EfficientNet: Rethinking Model Scaling for Convolutional Neural Networkscited by 6,442
- 2015Rethinking the Inception Architecture for Computer Visioncited by 5,475
- 2014Fully Convolutional Networks for Semantic Segmentationcited by 4,859
- 2017Squeeze-and-Excitation Networkscited by 4,789
- 2014Going Deeper with Convolutionscited by 4,241
- 2017MobileNets: Efficient Convolutional Neural Networks for Mobile Vision Applicationscited by 3,845
38 more
- 2016DeepLab: Semantic Image Segmentation with Deep Convolutional Nets, Atrous Convolution, and Fully Connected CRFscited by 3,241
- 2018Encoder-Decoder with Atrous Separable Convolution for Semantic Image Segmentationcited by 3,223
- 2016Aggregated Residual Transformations for Deep Neural Networkscited by 2,546
- 2016Pyramid Scene Parsing Networkcited by 2,479
- 2021Masked-attention Mask Transformer for Universal Image Segmentationcited by 2,351
- 2017Non-local Neural Networkscited by 2,293
- 2019RandAugment: Practical automated data augmentation with a reduced search spacecited by 2,269
- 2017Rethinking Atrous Convolution for Semantic Image Segmentationcited by 2,161
- 2018DARTS: Differentiable Architecture Searchcited by 1,798
- 2016Neural Architecture Search with Reinforcement Learningcited by 1,499
- 2022Model soups: averaging weights of multiple fine-tuned models improves accuracy without increasing inference timecited by 1,396
- 2020Rethinking Semantic Segmentation from a Sequence-to-Sequence Perspective with Transformerscited by 1,369
- 2021Per-Pixel Classification is Not All You Need for Semantic Segmentationcited by 1,152
- 2017Learning Transferable Architectures for Scalable Image Recognitioncited by 1,101
- 2018Dual Attention Network for Scene Segmentationcited by 1,003
- 2016Temporal Segment Networks: Towards Good Practices for Deep Action Recognitioncited by 902
- 2018TSM: Temporal Shift Module for Efficient Video Understandingcited by 856
- 2021Going deeper with Image Transformerscited by 764
- 2022InternImage: Exploring Large-Scale Vision Foundation Models with Deformable Convolutionscited by 482
- 2019Res2Net: A New Multi-scale Backbone Architecturecited by 470
- 2020ResNeSt: Split-Attention Networkscited by 445
- 2017Residual Attention Network for Image Classificationcited by 386
- 2019Fast AutoAugmentcited by 243
- 2021TrivialAugment: Tuning-free Yet State-of-the-Art Data Augmentationcited by 227
- 2017Large Kernel Matters -- Improve Semantic Segmentation by Global Convolutional Networkcited by 208
- 2016Wider or Deeper: Revisiting the ResNet Model for Visual Recognitioncited by 184
- 2015ParseNet: Looking Wider to See Bettercited by 177
- 2017Understanding Convolution for Semantic Segmentationcited by 168
- 2019Population Based Augmentation: Efficient Learning of Augmentation Policy Schedulescited by 154
- 2019Drop an Octave: Reducing Spatial Redundancy in Convolutional Neural Networks with Octave Convolutioncited by 140
- 2021Rethinking Architecture Selection in Differentiable NAScited by 92
- 2019Gate-Shift Networks for Video Action Recognitioncited by 76
- 2019Grouped Spatial-Temporal Aggregation for Efficient Action Recognitioncited by 55
- 2019Online Hyper-parameter Learning for Auto-Augmentation Strategycited by 38
- 2020UniformAugment: A Search-free Probabilistic Data Augmentation Approachcited by 38
- 2020Improving Auto-Augment via Augmentation-Wise Weight Sharingcited by 28
- 2016Fast, Exact and Multi-Scale Inference for Semantic Image Segmentation with Deep Gaussian CRFscited by 27
- 2014Feedforward semantic segmentation with zoom-out featurescited by 21
50 most cited of 55 All 55 (JSON)
Built on by
Not cited yet.