# Augmenting Supervised Neural Networks with Unsupervised Objectives for Large-scale Image Classification

arXiv: https://arxiv.org/abs/1606.06582 · HTML: https://mlsanity.com/paper/1606.06582/

## Key facts

- Cited by 18 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 21 Jun 2016.

## Built on by (most cited first, 18 of 18)

- [SupMAE: Supervised Masked Autoencoders Are Efficient Vision Learners](https://mlsanity.com/paper/2205.14540/) (2022)
- [Hebbian Semi-Supervised Learning in a Sample Efficiency Setting](https://mlsanity.com/paper/2103.09002/) (2021)
- [Understanding Invariance via Feedforward Inversion of Discriminatively Trained Classifiers](https://mlsanity.com/paper/2103.07470/) (2021)
- [FastHebb: Scaling Hebbian Training of Deep Neural Networks to ImageNet Level](https://mlsanity.com/paper/2207.03172/) (2022)
- [DejaVu: Conditional Regenerative Learning to Enhance Dense Prediction](https://mlsanity.com/paper/2303.01573/) (2023)
- [JR2net: A Joint Non-Linear Representation and Recovery Network for Compressive Spectral Imaging](https://mlsanity.com/paper/2205.07770/) (2022)
- [Deep Features for CBIR with Scarce Data using Hebbian Learning](https://mlsanity.com/paper/2205.08935/) (2022)
- [DECK: Model Hardening for Defending Pervasive Backdoors](https://mlsanity.com/paper/2206.09272/) (2022)
- [Biologically-inspired Semi-supervised Semantic Segmentation for Biomedical Imaging](https://mlsanity.com/paper/2412.03192/) (2024)
- [Streaming Self-Training via Domain-Agnostic Unlabeled Images](https://mlsanity.com/paper/2104.03309/) (2021)
- [Contextual Classification Using Self-Supervised Auxiliary Models for Deep Neural Networks](https://mlsanity.com/paper/2101.03057/) (2021)
- [AugmentedPCA: A Python Package of Supervised and Adversarial Linear Factor Models](https://mlsanity.com/paper/2201.02547/) (2022)
- [Fourier Transform Approximation as an Auxiliary Task for Image Classification](https://mlsanity.com/paper/2106.11478/) (2021)
- [Behaviour-conditioned policies for cooperative reinforcement learning tasks](https://mlsanity.com/paper/2110.01266/) (2021)
- [Learning to Assist Agents by Observing Them](https://mlsanity.com/paper/2110.01311/) (2021)
- [ExpertNet: A Symbiosis of Classification and Clustering](https://mlsanity.com/paper/2201.06344/) (2022)
- [DNN-GDITD: Out-of-distribution detection via Deep Neural Network based Gaussian Descriptor for Imbalanced Tabular Data](https://mlsanity.com/paper/2409.00980/) (2024)
- [Semi-Supervised Biomedical Image Segmentation via Diffusion Models and Teacher-Student Co-Training](https://mlsanity.com/paper/2504.01547/) (2025)

## Abstract (arXiv)

Unsupervised learning and supervised learning are key research topics in deep learning. However, as high-capacity supervised neural networks trained with a large amount of labels have achieved remarkable success in many computer vision tasks, the availability of large-scale labeled images reduced the significance of unsupervised learning. Inspired by the recent trend toward revisiting the importance of unsupervised learning, we investigate joint supervised and unsupervised learning in a large-scale setting by augmenting existing neural networks with decoding pathways for reconstruction. First, we demonstrate that the intermediate activations of pretrained large-scale classification networks preserve almost all the information of input images except a portion of local spatial details. Then, by end-to-end training of the entire augmented architecture with the reconstructive objective, we show improvement of the network performance for supervised tasks. We evaluate several variants of autoencoders, including the recently proposed "what-where" autoencoder that uses the encoder pooling switches, to study the importance of the architecture design. Taking the 16-layer VGGNet trained under the ImageNet ILSVRC 2012 protocol as a strong baseline for image classification, our methods improve the validation-set accuracy by a noticeable margin.