# Horizontal and Vertical Ensemble with Deep Representation for Classification

arXiv: https://arxiv.org/abs/1306.2759 · HTML: https://mlsanity.com/paper/1306.2759/

## Key facts

- Cited by 22 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 12 Jun 2013.

## Built on by (most cited first, 22 of 22)

- [Ensemble deep learning: A review](https://mlsanity.com/paper/2104.02395/) (2021)
- [Deep Ensembling with No Overhead for either Training or Testing: The All-Round Blessings of Dynamic Sparsity](https://mlsanity.com/paper/2106.14568/) (2021)
- [Sparse MoEs meet Efficient Ensembles](https://mlsanity.com/paper/2110.03360/) (2021)
- [Prune and Tune Ensembles: Low-Cost Ensemble Learning With Sparse Independent Subnetworks](https://mlsanity.com/paper/2202.11782/) (2022)
- [A Survey of Computer Vision Technologies In Urban and Controlled-environment Agriculture](https://mlsanity.com/paper/2210.11318/) (2022)
- [A Review of Bayesian Uncertainty Quantification in Deep Probabilistic Image Segmentation](https://mlsanity.com/paper/2411.16370/) (2024)
- [Sequential Bayesian Neural Subnetwork Ensembles](https://mlsanity.com/paper/2206.00794/) (2022)
- [Logit-Based Ensemble Distribution Distillation for Robust Autoregressive Sequence Uncertainties](https://mlsanity.com/paper/2305.10384/) (2023)
- [Boost Neural Networks by Checkpoints](https://mlsanity.com/paper/2110.00959/) (2021)
- [Explore the Potential Performance of Vision-and-Language Navigation Model: a Snapshot Ensemble Method](https://mlsanity.com/paper/2111.14267/) (2021)
- [Learning Proximal Operators to Discover Multiple Optima](https://mlsanity.com/paper/2201.11945/) (2022)
- [Instability in clinical risk stratification models using deep learning](https://mlsanity.com/paper/2211.10828/) (2022)
- [High-Dimensional Stock Portfolio Trading with Deep Reinforcement Learning](https://mlsanity.com/paper/2112.04755/) (2021)
- [Efficient Model Performance Estimation via Feature Histories](https://mlsanity.com/paper/2103.04450/) (2021)
- [Boosted Ensemble Learning based on Randomized NNs for Time Series Forecasting](https://mlsanity.com/paper/2203.00980/) (2022)
- [Towards a Systematic Approach to Design New Ensemble Learning Algorithms](https://mlsanity.com/paper/2402.06818/) (2024)
- [NeuSE: A Neural Snapshot Ensemble Method for Collaborative Filtering](https://mlsanity.com/paper/2104.07269/) (2021)
- [HCE: Improving Performance and Efficiency with Heterogeneously Compressed Neural Network Ensemble](https://mlsanity.com/paper/2301.07794/) (2023)
- [Relearning Forgotten Knowledge: on Forgetting, Overfit and Training-Free Ensembles of DNNs](https://mlsanity.com/paper/2310.11094/) (2023)
- [Forget Me Not: Fighting Local Overfitting with Knowledge Fusion and Distillation](https://mlsanity.com/paper/2507.08686/) (2025)
- [Symbolic Snapshot Ensembles](https://mlsanity.com/paper/2510.24633/) (2025)
- [Parameter Averaging in Link Prediction](https://mlsanity.com/paper/2510.25361/) (2025)

## Abstract (arXiv)

Representation learning, especially which by using deep learning, has been widely applied in classification. However, how to use limited size of labeled data to achieve good classification performance with deep neural network, and how can the learned features further improve classification remain indefinite. In this paper, we propose Horizontal Voting Vertical Voting and Horizontal Stacked Ensemble methods to improve the classification performance of deep neural networks. In the ICML 2013 Black Box Challenge, via using these methods independently, Bing Xu achieved 3rd in public leaderboard, and 7th in private leaderboard; Jingjing Xie achieved 4th in public leaderboard, and 5th in private leaderboard.