mlsanityData to 7 Oct 2026

ScaleNet: An Unsupervised Representation Learning Method for Limited Information

arXiv 2310.02386 ↗First posted 3 Oct 2023cs.CVcited by 0

Not on the map. Why?

Abstract (arXiv)

Although large-scale labeled data are essential for deep convolutional neural networks (ConvNets) to learn high-level semantic visual representations, it is time-consuming and impractical to collect and annotate large-scale datasets. A simple and efficient unsupervised representation learning method named ScaleNet based on multi-scale images is proposed in this study to enhance the performance of ConvNets when limited information is available. The input images are first resized to a smaller size and fed to the ConvNet to recognize the rotation degree. Next, the ConvNet learns the rotation-prediction task for the original size images based on the parameters transferred from the previous model. The CIFAR-10 and ImageNet datasets are examined on different architectures such as AlexNet and ResNet50 in this study. The current study demonstrates that specific image features, such as Harris corner information, play a critical role in the efficiency of the rotation-prediction task. The ScaleNet supersedes the RotNet by ~7% in the limited CIFAR-10 dataset. The transferred parameters from a ScaleNet model with limited data improve the ImageNet Classification task by about 6% compared to the RotNet model. This study shows the capability of the ScaleNet method to improve other cutting-edge models such as SimCLR by learning effective features for classification tasks.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 34 of its 52 references that are in the map. First posted on arXiv 3 Oct 2023.

Builds on

34 of 52 references.

References · 34

  1. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  2. 2015U-Net: Convolutional Networks for Biomedical Image Segmentationcited by 19,604
  3. 2020A Simple Framework for Contrastive Learning of Visual Representationscited by 11,979
  4. 2015Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networkscited by 9,617
  5. 2018Representation Learning with Contrastive Predictive Codingcited by 7,662
  6. 2019Momentum Contrast for Unsupervised Visual Representation Learningcited by 7,482
  7. 2020Bootstrap your own latent: A new approach to self-supervised Learningcited by 4,712
  8. 2015Fast R-CNNcited by 2,930
  9. 2020Exploring Simple Siamese Representation Learningcited by 2,853
  10. 2020Unsupervised Learning of Visual Features by Contrasting Cluster Assignmentscited by 2,845
  11. 2019ALBERT: A Lite BERT for Self-supervised Learning of Language Representationscited by 2,772
  12. 2013Rich feature hierarchies for accurate object detection and semantic segmentationcited by 2,160
22 more
  1. 2018Unsupervised Feature Learning via Non-Parametric Instance-level Discriminationcited by 1,838
  2. 2018Unsupervised Representation Learning by Predicting Image Rotationscited by 1,586
  3. 2016Unsupervised Learning of Visual Representations by Solving Jigsaw Puzzlescited by 1,379
  4. 2014How transferable are features in deep neural networks?cited by 1,216
  5. 2019A Comprehensive Survey on Transfer Learningcited by 1,154
  6. 2016Colorful Image Colorizationcited by 1,113
  7. 2015Unsupervised Visual Representation Learning by Context Predictioncited by 1,062
  8. 2018Deep Clustering for Unsupervised Learning of Visual Featurescited by 1,020
  9. 2014Deep Visual-Semantic Alignments for Generating Image Descriptionscited by 966
  10. 2017Large Batch Training of Convolutional Networkscited by 529
  11. 2019Self-labelling via simultaneous clustering and representation learningcited by 525
  12. 2019Revisiting Self-Supervised Visual Representation Learningcited by 250
  13. 2018GAN-based Synthetic Medical Image Augmentation for increased CNN Performance in Liver Lesion Classificationcited by 242
  14. 2017Representation Learning by Learning to Countcited by 120
  15. 2018Data Augmentation by Pairing Samples for Images Classificationcited by 96
  16. 2018Self-Supervised GANs via Auxiliary Rotation Losscited by 94
  17. 2018Self-Supervised Spatiotemporal Feature Learning via Video Rotation Predictioncited by 71
  18. 2016Going Deeper with Contextual CNN for Hyperspectral Image Classificationcited by 58
  19. 2016Classification With an Edge: Improving Semantic Image Segmentation with Boundary Detectioncited by 38
  20. 2020Steering Self-Supervised Feature Learning Beyond Local Pixel Statisticscited by 19
  21. 2015Deep Image: Scaling up Image Recognitioncited by 16
  22. 2016Frankenstein: Learning Deep Face Representations using Small Datacited by 7

Built on by

Not cited yet.