# Single Model Ensemble using Pseudo-Tags and Distinct Vectors

arXiv: https://arxiv.org/abs/2005.00879 · HTML: https://mlsanity.com/paper/2005.00879/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 2 May 2020.

## Built on by (most cited first, 1 of 1)

- [Single Model Ensemble for Subword Regularized Models in Low-Resource Machine Translation](https://mlsanity.com/paper/2203.13528/) (2022)

## Abstract (arXiv)

Model ensemble techniques often increase task performance in neural networks; however, they require increased time, memory, and management effort. In this study, we propose a novel method that replicates the effects of a model ensemble with a single model. Our approach creates K-virtual models within a single parameter space using K-distinct pseudo-tags and K-distinct vectors. Experiments on text classification and sequence labeling tasks on several datasets demonstrate that our method emulates or outperforms a traditional model ensemble with 1/K-times fewer parameters.