# OSLNet: Deep Small-Sample Classification with an Orthogonal Softmax Layer

arXiv: https://arxiv.org/abs/2004.09033 · HTML: https://mlsanity.com/paper/2004.09033/

## Key facts

- Cited by 7 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 20 Apr 2020.

## Built on by (most cited first, 7 of 7)

- [Maximum Class Separation as Inductive Bias in One Matrix](https://mlsanity.com/paper/2206.08704/) (2022)
- [Avoiding Forgetting and Allowing Forward Transfer in Continual Learning via Sparse Networks](https://mlsanity.com/paper/2110.05329/) (2021)
- [Data-Efficient Learning via Minimizing Hyperspherical Energy](https://mlsanity.com/paper/2206.15204/) (2022)
- [Differential-UMamba: Rethinking Tumor Segmentation Under Limited Data Scenarios](https://mlsanity.com/paper/2507.18177/) (2025)
- [Face-Voice Association with Inductive Bias for Maximum Class Separation](https://mlsanity.com/paper/2601.13651/) (2026)
- [Clue Me In: Semi-Supervised FGVC with Out-of-Distribution Data](https://mlsanity.com/paper/2112.02825/) (2021)
- [On Training Targets and Activation Functions for Deep Representation Learning in Text-Dependent Speaker Verification](https://mlsanity.com/paper/2201.06426/) (2022)

## Abstract (arXiv)

A deep neural network of multiple nonlinear layers forms a large function space, which can easily lead to overfitting when it encounters small-sample data. To mitigate overfitting in small-sample classification, learning more discriminative features from small-sample data is becoming a new trend. To this end, this paper aims to find a subspace of neural networks that can facilitate a large decision margin. Specifically, we propose the Orthogonal Softmax Layer (OSL), which makes the weight vectors in the classification layer remain orthogonal during both the training and test processes. The Rademacher complexity of a network using the OSL is only frac1K, where K is the number of classes, of that of a network using the fully connected classification layer, leading to a tighter generalization error bound. Experimental results demonstrate that the proposed OSL has better performance than the methods used for comparison on four small-sample benchmark datasets, as well as its applicability to large-sample datasets. Codes are available at: https://github.com/dongliangchang/OSLNet.