mlsanityData to 7 Oct 2026

Ensemble deep learning: A review

arXiv 2104.02395 ↗First posted 6 Apr 2021cs.LG cs.AI cs.CVcited by 283

Not on the map. Why?

Abstract (arXiv)

Ensemble learning combines several individual models to obtain better generalization performance. Currently, deep learning architectures are showing better performance compared to the shallow or traditional models. Deep ensemble learning models combine the advantages of both the deep learning models as well as the ensemble learning such that the final model has better generalization performance. This paper reviews the state-of-art deep ensemble models and hence serves as an extensive summary for the researchers. The ensemble models are broadly categorised into bagging, boosting, stacking, negative correlation based deep ensemble models, explicit/implicit ensembles, homogeneous/heterogeneous ensemble, decision fusion strategies based deep ensemble models. Applications of deep ensemble models in different domains are also briefly discussed. Finally, we conclude this paper with some potential future research directions.

Key facts. Cited by 283 papers in the map (data through 7 Oct 2026). Builds on 41 of its 249 references that are in the map. First posted on arXiv 6 Apr 2021.

Builds on

41 of 249 references.

References · 41

  1. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  2. 2014Very Deep Convolutional Networks for Large-Scale Image Recognitioncited by 15,267
  3. 2015Distilling the Knowledge in a Neural Networkcited by 10,076
  4. 2014ImageNet Large Scale Visual Recognition Challengecited by 8,441
  5. 2014Going Deeper with Convolutionscited by 4,231
  6. 2013Playing Atari with Deep Reinforcement Learningcited by 3,211
  7. 2014DeepWalk: Online Learning of Social Representationscited by 1,543
  8. 2016Temporal Ensembling for Semi-Supervised Learningcited by 1,054
  9. 2016Deep Networks with Stochastic Depthcited by 973
  10. 2017Snapshot Ensembles: Train 1, get M for freecited by 349
  11. 2014The Loss Surfaces of Multilayer Networkscited by 278
  12. 2019Deep Stacked Hierarchical Multi-patch Network for Image Deblurringcited by 205
29 more
  1. 2015Training Very Deep Networkscited by 172
  2. 2016AdaNet: Adaptive Structural Learning of Artificial Neural Networkscited by 54
  3. 2017The Relative Performance of Ensemble Methods with Deep Convolutional Neural Networks for Image Classificationcited by 50
  4. 2017Deep Forestcited by 46
  5. 2017Learning Deep ResNet Blocks Sequentially using Boosting Theorycited by 32
  6. 2013Horizontal and Vertical Ensemble with Deep Representation for Classificationcited by 22
  7. 2016Locally Weighted Ensemble Clusteringcited by 19
  8. 2016Swapout: Learning an ensemble of deep architecturescited by 19
  9. 2017Incremental Boosting Convolutional Neural Network for Facial Action Unit Recognitioncited by 16
  10. 2020Generalized Negative Correlation Learning for Deep Ensemblingcited by 16
  11. 2015Online Gradient Boostingcited by 15
  12. 2016A Deep Learning Approach to Unsupervised Ensemble Learningcited by 12
  13. 2019Ensemble-Based Deep Reinforcement Learning for Chatbotscited by 11
  14. 2019EnAET: A Self-Trained framework for Semi-Supervised and Supervised Learning with Ensemble Transformationscited by 11
  15. 2015Gradual DropIn of Layers to Train Very Deep Neural Networkscited by 8
  16. 2019Particle Swarm Optimisation for Evolving Deep Neural Networks for Image Classification by Evolving and Stacking Transferable Blockscited by 8
  17. 2017Online Learning for Offloading and Autoscaling in Energy Harvesting Mobile Edge Computingcited by 7
  18. 2017Deep Incremental Boostingcited by 7
  19. 2018Stacking-Based Deep Neural Network: Deep Analytic Network for Pattern Classificationcited by 7
  20. 2016Deep Neural Ensemble for Retinal Vessel Segmentation in Fundus Images towards Achieving Label-free Angiographycited by 5
  21. 2019SVM-based Deep Stacking Networkscited by 5
  22. 2019Stacked Autoencoder Based Deep Random Vector Functional Link Neural Network for Classificationcited by 5
  23. 2020MNIST-NET10: A heterogeneous deep networks fusion based on the degree of certainty to reach 0.1 error rate. Ensembles overview and proposalcited by 5
  24. 2019Unsupervised Feature Learning with K-means and An Ensemble of Deep Convolutional Neural Networks for Medical Image Classificationcited by 4
  25. 2017Propensity score prediction for electronic healthcare databases using Super Learner and High-dimensional Propensity Score Methodscited by 3
  26. 2019Residual Networks Behave Like Boosting Algorithmscited by 3
  27. 2021Deep Ensemble Learning-based Approach to Real-time Power System State Estimationcited by 3
  28. 2014Multi-label ensemble based on variable pairwise constraint projectioncited by 1
  29. 2015Sparse Deep Stacking Network for Image Classificationcited by 1

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 12; 2022: 50; 2023: 58; 2024: 79; 2025: 64; 2026: 20.

283 papers.

Sort:

By area:

  1. 2023SpecInfer: Accelerating Generative Large Language Model Serving with Tree-based Speculative Inference and Verificationcited by 359
  2. 2025A Comprehensive Survey of Reward Models: Taxonomy, Applications, Challenges, and Futurecited by 53
  3. 2023Fusing Models with Complementary Expertisecited by 45
  4. 2023A review of uncertainty quantification in medical image analysis: probabilistic and non-probabilistic methodscited by 39
  5. 2023Boosted Prompt Ensembles for Large Language Modelscited by 36
  6. 2022Forecast combinations: an over 50-year reviewcited by 34
  7. 2022Random vector functional link network: recent developments, applications, and future directionscited by 33
  8. 2022LidarCLIP or: How I Learned to Talk to Point Cloudscited by 33
  9. 2024TIP: Tabular-Image Pre-training for Multimodal Classification with Incomplete Datacited by 32
  10. 2024Co-Supervised Learning: Improving Weak-to-Strong Generalization with Hierarchical Mixture of Expertscited by 28
  11. 2022Assessing generalisability of deep learning-based polyp detection and segmentation methods through a computer vision challengecited by 27
  12. 2023A review of ensemble learning and data augmentation models for class imbalanced problems: combination, implementation and evaluationcited by 24
38 more
  1. 2024Attention Prompting on Image for Large Vision-Language Modelscited by 23
  2. 2022Long-Tail Prediction Uncertainty Aware Trajectory Planning for Self-driving Vehiclescited by 22
  3. 2023Towards long-tailed, multi-label disease classification from chest X-ray: Overview of the CXR-LT challengecited by 22
  4. 2024Dipper: Diversity in Prompts for Producing Large Language Model Ensembles in Reasoning taskscited by 22
  5. 2023Sparse Model Soups: A Recipe for Improved Pruning via Model Averagingcited by 20
  6. 2024Emergent Equivariance in Deep Ensemblescited by 18
  7. 2022STAP: Sequencing Task-Agnostic Policiescited by 17
  8. 2021Comprehensive Review On Twin Support Vector Machinescited by 16
  9. 2023Ensemble Reinforcement Learning: A Surveycited by 16
  10. 2024MixedNUTS: Training-Free Accuracy-Robustness Balance via Nonlinearly Mixed Classifierscited by 16
  11. 2024PLeaS -- Merging Models with Permutations and Least Squarescited by 16
  12. 2024Estimating Canopy Height at Scalecited by 15
  13. 2022Learning useful representations for shifting tasks and distributionscited by 14
  14. 2024Self-Ensembling Gaussian Splatting for Few-Shot Novel View Synthesiscited by 14
  15. 2022Deep Learning for Brain Age Estimation: A Systematic Reviewcited by 12
  16. 2024FREE-Merging: Fourier Transform for Efficient Model Mergingcited by 12
  17. 2022PSO-PINN: Physics-Informed Neural Networks Trained with Particle Swarm Optimizationcited by 11
  18. 2022Machine Learning and Deep Learning -- A review for Ecologistscited by 11
  19. 2023Verifying Generalization in Deep Learningcited by 11
  20. 2022Enhancing Heterogeneous Federated Learning with Knowledge Extraction and Multi-Model Fusioncited by 10
  21. 2023A generative artificial intelligence framework based on a molecular diffusion model for the design of metal-organic frameworks for carbon capturecited by 10
  22. 2024RACER: Epistemic Risk-Sensitive RL Enables Fast Driving with Fewer Crashescited by 10
  23. 2025CXR-LT 2024: A MICCAI challenge on long-tailed, multi-label, and zero-shot disease classification from chest X-raycited by 10
  24. 2021Generalisation effects of predictive uncertainty estimation in deep learning for digital pathologycited by 9
  25. 2022Evaluation of Thermal Imaging on Embedded GPU Platforms for Application in Vehicular Assistance Systemscited by 9
  26. 2022C^*-algebra Net: A New Approach Generalizing Neural Network Parameters to C^*-algebracited by 9
  27. 2023Sebis at SemEval-2023 Task 7: A Joint System for Natural Language Inference and Evidence Retrieval from Clinical Trial Reportscited by 9
  28. 2023Applications of Spiking Neural Networks in Visual Place Recognitioncited by 9
  29. 2024The VoxCeleb Speaker Recognition Challenge: A Retrospectivecited by 9
  30. 2025Scalable Model Merging with Progressive Layer-wise Distillationcited by 9
  31. 2025An Axiomatic Assessment of Entropy- and Variance-based Uncertainty Quantification in Regressioncited by 9
  32. 2022An Analysis on Ensemble Learning optimized Medical Image Classification with Deep Convolutional Neural Networkscited by 8
  33. 2022Efficient Adaptive Ensembling for Image Classificationcited by 8
  34. 2023Bootstrap aggregation and confidence measures to improve time series causal discoverycited by 8
  35. 2023DNNShifter: An Efficient DNN Pruning System for Edge Computingcited by 8
  36. 2026Symphony: A Cognitively-Inspired Multi-Agent System for Long-Video Understandingcited by 8
  37. 2022Robust Ensemble Morph Detection with Domain Generalizationcited by 7
  38. 2022Thunderstorm nowcasting with deep learning: a multi-hazard data fusion modelcited by 7

50 most cited of 283 All 283 (JSON)