# Semi-Supervised Learning for Text Classification by Layer Partitioning

arXiv: https://arxiv.org/abs/1911.11756 · HTML: https://mlsanity.com/paper/1911.11756/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 26 Nov 2019.

## Built on by (most cited first, 3 of 3)

- [Uncertainty-aware Parameter-Efficient Self-training for Semi-supervised Language Understanding](https://mlsanity.com/paper/2310.13022/) (2023)
- [Layer-Wise Partitioning and Merging for Efficient and Scalable Deep Learning](https://mlsanity.com/paper/2207.11019/) (2022)
- [CoVLM: Leveraging Consensus from Vision-Language Models for Semi-supervised Multi-modal Fake News Detection](https://mlsanity.com/paper/2410.04426/) (2024)

## Abstract (arXiv)

Most recent neural semi-supervised learning algorithms rely on adding small perturbation to either the input vectors or their representations. These methods have been successful on computer vision tasks as the images form a continuous manifold, but are not appropriate for discrete input such as sentence. To adapt these methods to text input, we propose to decompose a neural network M into two components F and U so that M = Ucirc F. The layers in F are then frozen and only the layers in U will be updated during most time of the training. In this way, F serves as a feature extractor that maps the input to high-level representation and adds systematical noise using dropout. We can then train U using any state-of-the-art SSL algorithms such as Pi-model, temporal ensembling, mean teacher, etc. Furthermore, this gradually unfreezing schedule also prevents a pretrained model from catastrophic forgetting. The experimental results demonstrate that our approach provides improvements when compared to state of the art methods especially on short texts.