# Training Group Orthogonal Neural Networks with Privileged Information

arXiv: https://arxiv.org/abs/1701.06772 · HTML: https://mlsanity.com/paper/1701.06772/

## Key facts

- Cited by 6 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 24 Jan 2017.

## Built on by (most cited first, 6 of 6)

- [Toward Understanding Privileged Features Distillation in Learning-to-Rank](https://mlsanity.com/paper/2209.08754/) (2022)
- [Group-wise Inhibition based Feature Regularization for Robust Classification](https://mlsanity.com/paper/2103.02152/) (2021)
- [Feedback Gradient Descent: Efficient and Stable Optimization with Orthogonality for DNNs](https://mlsanity.com/paper/2205.08385/) (2022)
- [Deep Diversity-Enhanced Feature Representation of Hyperspectral Images](https://mlsanity.com/paper/2301.06132/) (2023)
- [Unsupervised domain adaptation by learning using privileged information](https://mlsanity.com/paper/2303.09350/) (2023)
- [Neglected Free Lunch -- Learning Image Classifiers Using Annotation Byproducts](https://mlsanity.com/paper/2303.17595/) (2023)

## Abstract (arXiv)

Learning rich and diverse representations is critical for the performance of deep convolutional neural networks (CNNs). In this paper, we consider how to use privileged information to promote inherent diversity of a single CNN model such that the model can learn better representations and offer stronger generalization ability. To this end, we propose a novel group orthogonal convolutional neural network (GoCNN) that learns untangled representations within each layer by exploiting provided privileged information and enhances representation diversity effectively. We take image classification as an example where image segmentation annotations are used as privileged information during the training process. Experiments on two benchmark datasets -- ImageNet and PASCAL VOC -- clearly demonstrate the strong generalization ability of our proposed GoCNN model. On the ImageNet dataset, GoCNN improves the performance of state-of-the-art ResNet-152 model by absolute value of 1.2% while only uses privileged information of 10% of the training images, confirming effectiveness of GoCNN on utilizing available privileged knowledge to train better CNNs.