mlsanityData to 14 Oct 2026

On Margins and Generalisation for Voting Classifiers

arXiv 2206.04607 ↗First posted 9 Jun 2022cs.LG math.ST stat.MLcited by 7

Not on the map. Why?

Abstract (arXiv)

We study the generalisation properties of majority voting on finite ensembles of classifiers, proving margin-based generalisation bounds via the PAC-Bayes theory. These provide state-of-the-art guarantees on a number of classification tasks. Our central results leverage the Dirichlet posteriors studied recently by Zantedeschi et al. [2021] for training voting classifiers; in contrast to that work our bounds apply to non-randomised votes via the use of margins. Our contributions add perspective to the debate on the "margins theory" proposed by Schapire et al. [1998] for the generalisation of ensemble classifiers.

Key facts. Cited by 7 papers in the map (data through 14 Oct 2026). Builds on 25 of its 57 references that are in the map. First posted on arXiv 9 Jun 2022.

Builds on

25 of 57 references.

References · 25

  1. 2014Adam: A Method for Stochastic Optimizationcited by 37,240
  2. 2016XGBoost: A Scalable Tree Boosting Systemcited by 5,166
  3. 2017Fashion-MNIST: a Novel Image Dataset for Benchmarking Machine Learning Algorithmscited by 4,349
  4. 2021Tabular Data: Deep Learning is Not All You Needcited by 535
  5. 2017Computing Nonvacuous Generalization Bounds for Deep (Stochastic) Neural Networks with Many More Parameters than Training Datacited by 533
  6. 2021User-friendly introduction to PAC-Bayes boundscited by 239
  7. 2019A Primer on PAC-Bayesian Learningcited by 132
  8. 2018Non-Vacuous Generalization Bounds at the ImageNet Scale: A PAC-Bayesian Compression Approachcited by 126
  9. 2020Tighter risk certificates for neural networkscited by 112
  10. 2017Entropy-SGD optimizes the prior of a PAC-Bayes bound: Generalization properties of Entropy-SGD and data-dependent priorscited by 81
  11. 2020On the role of data in PAC-Bayes boundscited by 64
  12. 2016A Strongly Quasiconvex PAC-Bayesian Boundcited by 54
13 more
  1. 2015Risk Bounds for the Majority Vote: From a PAC-Bayesian Analysis to a Learning Algorithmcited by 50
  2. 2019Dichotomize and Generalize: PAC-Bayesian Binary Activated Deep Neural Networkscited by 38
  3. 2020Second Order PAC-Bayesian Bounds for the Weighted Majority Votecited by 34
  4. 2020Differentiable PAC-Bayes Objectives with Partially Aggregated Neural Networkscited by 29
  5. 2021How Tight Can PAC-Bayes be in the Small Data Regime?cited by 24
  6. 2022Non-Vacuous Generalisation Bounds for Shallow Neural Networkscited by 22
  7. 2021On Margins and Derandomisation in PAC-Bayescited by 19
  8. 2021Learning Stochastic Majority Votes by Minimizing a PAC-Bayes Generalization Boundcited by 16
  9. 2021Chebyshev-Cantelli PAC-Bayes-Bennett Inequality for the Weighted Majority Votecited by 13
  10. 2020Spacecraft Collision Avoidance Challenge: design and results of a machine learning competitioncited by 11
  11. 2018On PAC-Bayesian Bounds for Random Forestscited by 10
  12. 2020Margins are Insufficient for Explaining Gradient Boostingcited by 10
  13. 2021Self-Bounding Majority Vote Learning Algorithms by the Direct Minimization of a Tight PAC-Bayesian C-Boundcited by 8

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2022: 2; 2023: 4; 2024: 0; 2025: 0; 2026: 1.

7 papers.

  1. 2023MMD-FUSE: Learning and Combining Kernels for Two-Sample Testing Without Data Splittingcited by 33
  2. 2022PAC-Bayes Generalisation Bounds for Heavy-Tailed Losses through Supermartingalescited by 24
  3. 2023Learning via Wasserstein-Based High Probability Generalisation Boundscited by 12
  4. 2022Tighter PAC-Bayes Generalisation Bounds by Leveraging Example Difficultycited by 9
  5. 2023Are Ensembles Getting Better all the Time?cited by 6
  6. 2023Federated Learning with Nonvacuous Generalisation Boundscited by 2
  7. 2026On the disintegration of the stochastic majority vote: From PAC-Bayesian bounds to a self-bounding algorithmcited by 0