Spatial Transformer Networks for Curriculum Learning
Not on the map. Why?
Abstract (arXiv)
Curriculum learning is a bio-inspired training technique that is widely adopted to machine learning for improved optimization and better training of neural networks regarding the convergence rate or obtained accuracy. The main concept in curriculum learning is to start the training with simpler tasks and gradually increase the level of difficulty. Therefore, a natural question is how to determine or generate these simpler tasks. In this work, we take inspiration from Spatial Transformer Networks (STNs) in order to form an easy-to-hard curriculum. As STNs have been proven to be capable of removing the clutter from the input images and obtaining higher accuracy in image classification tasks, we hypothesize that images processed by STNs can be seen as easier tasks and utilized in the interest of curriculum learning. To this end, we study multiple strategies developed for shaping the training curriculum, using the data generated by STNs. We perform various experiments on cluttered MNIST and Fashion-MNIST datasets, where on the former, we obtain an improvement of 3.8pp in classification accuracy compared to the baseline.
Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 17 of its 39 references that are in the map. First posted on arXiv 22 Aug 2021.
Builds on
17 of 39 references.
References · 17
- 2014Adam: A Method for Stochastic Optimizationcited by 37,240
- 2015Distilling the Knowledge in a Neural Networkcited by 10,324
- 2015Spatial Transformer Networkscited by 1,429
- 2015Scheduled Sampling for Sequence Prediction with Recurrent Neural Networkscited by 803
- 2017MentorNet: Learning Data-Driven Curriculum for Very Deep Neural Networks on Corrupted Labelscited by 653
- 2016Training Region-based Object Detectors with Online Hard Example Miningcited by 579
- 2014Recurrent Models of Visual Attentioncited by 349
- 2019On The Power of Curriculum Learning in Training Deep Networkscited by 286
- 2017Automated Curriculum Learning for Neural Networkscited by 257
- 2017Teacher-Student Curriculum Learningcited by 210
- 2017Reverse Curriculum Generation for Reinforcement Learningcited by 198
- 2017Active Bias: Training More Accurate Neural Networks by Emphasizing High Variance Samplescited by 147
5 more
- 2018Curriculum Learning by Transfer Learning: Theory and Experiments with Deep Networkscited by 114
- 2014Learning to Executecited by 106
- 2019Dynamic Curriculum Learning for Imbalanced Data Classificationcited by 93
- 2020Hybrid-S2S: Video Object Segmentation with Recurrent Networks and Correspondence Matchingcited by 1
- 2021A Reinforcement Learning Approach for Sequential Spatial Transformer Networkscited by 1
Built on by
Not cited yet.