mlsanityData to 7 Oct 2026

Generalization In Multi-Objective Machine Learning

arXiv 2208.13499 ↗First posted 29 Aug 2022cs.LG cs.AI stat.MLcited by 4

Not on the map. Why?

Abstract (arXiv)

Modern machine learning tasks often require considering not just one but multiple objectives. For example, besides the prediction quality, this could be the efficiency, robustness or fairness of the learned models, or any of their combinations. Multi-objective learning offers a natural framework for handling such problems without having to commit to early trade-offs. Surprisingly, statistical learning theory so far offers almost no insight into the generalization properties of multi-objective learning. In this work, we make first steps to fill this gap: we establish foundational generalization bounds for the multi-objective setting as well as generalization and excess bounds for learning with scalarizations. We also provide the first theoretical analysis of the relation between the Pareto-optimal sets of the true objectives and the Pareto-optimal sets of their empirical approximations from training data. In particular, we show a surprising asymmetry: all Pareto-optimal solutions can be approximated by empirically Pareto-optimal ones, but not vice versa.

Key facts. Cited by 4 papers in the map (data through 7 Oct 2026). Builds on 31 of its 96 references that are in the map. First posted on arXiv 29 Aug 2022.

Builds on

31 of 96 references.

References · 31

  1. 2017Towards Deep Learning Models Resistant to Adversarial Attackscited by 6,326
  2. 2016Equality of Opportunity in Supervised Learningcited by 2,000
  3. 2018Multi-Task Learning as Multi-Objective Optimizationcited by 809
  4. 2021A Survey of Quantization Methods for Efficient Neural Network Inferencecited by 798
  5. 2016Fair prediction with disparate impact: A study of bias in recidivism prediction instrumentscited by 671
  6. 2017BranchyNet: Fast Inference via Early Exiting from Deep Neural Networkscited by 591
  7. 2016Inherent Trade-Offs in the Fair Determination of Risk Scorescited by 582
  8. 2015Train faster, generalize better: Stability of stochastic gradient descentcited by 547
  9. 2017Computing Nonvacuous Generalization Bounds for Deep (Stochastic) Neural Networks with Many More Parameters than Training Datacited by 527
  10. 2021Sparsity in Deep Learning: Pruning and growth for efficient inference and training in neural networkscited by 521
  11. 2017Fairness in Criminal Justice Risk Assessments: The State of the Artcited by 315
  12. 2017Parseval Networks: Improving Robustness to Adversarial Examplescited by 288
19 more
  1. 2018Towards Fast Computation of Certified Robustness for ReLU Networkscited by 254
  2. 2019Pareto Multi-Task Learningcited by 224
  3. 2021Efficient Deep Learning: A Survey on Making Deep Learning Models Smaller, Faster, and Bettercited by 153
  4. 2020Minimax Pareto Fairness: A Multi Objective Perspectivecited by 152
  5. 2018Rademacher Complexity for Adversarially Robust Generalizationcited by 132
  6. 2020Learning the Pareto Front with Hypernetworkscited by 122
  7. 2020Tilted Empirical Risk Minimizationcited by 103
  8. 2020Efficient Continuous Pareto Exploration in Multi-Task Learningcited by 57
  9. 2021Scalable Pareto Front Approximation for Deep Multi-Objective Learningcited by 45
  10. 2018Multi-objective Evolutionary Federated Learningcited by 42
  11. 2020Accuracy and Fairness Trade-offs in Machine Learning: A Stochastic Multi-Objective Approachcited by 39
  12. 2021Multi-Objective Meta Learningcited by 39
  13. 2020The Fairness-Accuracy Pareto Frontcited by 26
  14. 2020Addressing Fairness in Classification with a Model-Agnostic Multi-Objective Algorithmcited by 23
  15. 2021Fairness-Aware PAC Learning from Corrupted Datacited by 18
  16. 2016Algorithms for Fitting the Constrained Lassocited by 17
  17. 2021Multi-Objective Learning to Predict Pareto Fronts Using Hypervolume Maximizationcited by 16
  18. 2021Pareto Efficient Fairness in Supervised Learning: From Extraction to Tracingcited by 8
  19. 2021Multi-task problems are not multi-objectivecited by 3

Built on by

Cited by year

Citing papers per year: 2023: 1; 2024: 1; 2025: 2.

4 papers.

  1. 2023Three-Way Trade-Off in Multi-Objective Learning: Optimization, Generalization and Conflict-Avoidancecited by 35
  2. 2025Gradient-Based Multi-Objective Deep Learning: Algorithms, Theories, Applications, and Beyondcited by 15
  3. 2024Towards Stable Machine Learning Model Retraining via Slowly Varying Sequencescited by 2
  4. 2025Learning Pareto manifolds in high dimensions: How can regularization help?cited by 1