mlsanityData to 7 Oct 2026

H-Consistency Guarantees for Regression

arXiv 2403.19480 ↗First posted 28 Mar 2024cs.LG stat.MLcited by 21

Not on the map. Why?

Abstract (arXiv)

We present a detailed study of H-consistency bounds for regression. We first present new theorems that generalize the tools previously given to establish H-consistency bounds. This generalization proves essential for analyzing H-consistency bounds specific to regression. Next, we prove a series of novel H-consistency bounds for surrogate loss functions of the squared loss, under the assumption of a symmetric distribution and a bounded hypothesis set. This includes positive results for the Huber loss, all ℓp losses, p ≥ 1, the squared ε-insensitive loss, as well as a negative result for the ε-insensitive loss used in squared Support Vector Regression (SVR). We further leverage our analysis of H-consistency for regression and derive principled surrogate losses for adversarial regression (Section 5). This readily establishes novel algorithms for adversarial regression, for which we report favorable experimental results in Section 6.

Key facts. Cited by 21 papers in the map (data through 7 Oct 2026). Builds on 37 of its 74 references that are in the map. First posted on arXiv 28 Mar 2024.

Builds on

37 of 74 references.

References · 37

  1. 2017Towards Deep Learning Models Resistant to Adversarial Attackscited by 6,326
  2. 2013Intriguing properties of neural networkscited by 5,114
  3. 2014Sequence to Sequence Learning with Neural Networkscited by 2,974
  4. 2019Theoretically Principled Trade-off between Robustness and Accuracycited by 1,510
  5. 2018Robustness May Be at Odds with Accuracycited by 873
  6. 2016CVXPY: A Python-Embedded Modeling Language for Convex Optimizationcited by 633
  7. 2018Regularisation of Neural Networks by Enforcing Lipschitz Continuitycited by 311
  8. 2023Cross-Entropy Loss Functions: Theoretical Analysis and Applicationscited by 206
  9. 2017Formal Guarantees on the Robustness of a Classifier against Adversarial Manipulationcited by 182
  10. 2019Adversarial Training Can Hurt Generalizationcited by 131
  11. 2020Precise Tradeoffs in Adversarial Training for Linear Regressioncited by 76
  12. 2020Precise Statistical Analysis of Classification Accuracies for Adversarial Trainingcited by 49
25 more
  1. 2021Calibration and Consistency of Adversarial Surrogate Lossescited by 47
  2. 2019Scaleable input gradient regularization for adversarial robustnesscited by 45
  3. 2022The curse of overparametrization in adversarial training: Precise analysis of robust generalization for random features regressioncited by 40
  4. 2020The Curious Case of Adversarially Robust Models: More Data Can Help, Double Descend, or Hurt Generalizationcited by 39
  5. 2023Principled Approaches for Learning to Defer with Multiple Expertscited by 38
  6. 2023Predictor-Rejector Multi-Class Abstention: Theoretical Analysis and Algorithmscited by 36
  7. 2016A Consistent Regularization Approach for Structured Predictioncited by 33
  8. 2023Learning to Reject with a Fixed Predictor: Application to Decontextualizationcited by 32
  9. 2023Theoretically Grounded Loss Functions and Algorithms for Score-Based Multi-Class Abstentioncited by 32
  10. 2023Revisiting Discriminative vs. Generative Classifiers: Theory and Implicationscited by 29
  11. 2023Ranking with Abstentioncited by 27
  12. 2021A Finer Calibration Analysis for Adversarial Robustnesscited by 26
  13. 2014On the Consistency of Ordinal Regression Methodscited by 24
  14. 2017On Structured Prediction Theory with Calibrated Convex Surrogate Lossescited by 23
  15. 2020Adversarially Robust Estimate and Risk Analysis in Linear Regressioncited by 22
  16. 2019An Embedding Framework for Consistent Polyhedral Surrogatescited by 21
  17. 2019Structured Prediction with Projection Oraclescited by 21
  18. 2022Overparameterized Linear Regression under Adversarial Attackscited by 21
  19. 2020Convex Calibrated Surrogates for the Multi-Label F-Measurecited by 18
  20. 2023Regularization properties of adversarially-trained linear regressioncited by 17
  21. 2021Surrogate Regret Bounds for Polyhedral Lossescited by 15
  22. 2022A Non-Asymptotic Moreau Envelope Theory for High-Dimensional Generalized Linear Modelscited by 11
  23. 2020Weston-Watkins Hinge Loss and Ordered Partitionscited by 9
  24. 2023On Classification-Calibration of Gamma-Phi Lossescited by 6
  25. 2023Non-Asymptotic Bounds for Adversarial Excess Risk under Misspecified Modelscited by 2

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 7; 2025: 9; 2026: 5.

21 papers.

Sort:

  1. 2024Realizable H-Consistent and Bayes-Consistent Loss Functions for Learning to Defercited by 26
  2. 2024A Universal Growth Rate for Learning with Smooth Surrogate Lossescited by 24
  3. 2024Cardinality-Aware Set Prediction and Top-k Classificationcited by 23
  4. 2024Multi-Label Learning with Stronger Consistency Guaranteescited by 22
  5. 2024Enhanced H-Consistency Boundscited by 18
  6. 2024A Two-Stage Learning-to-Defer Approach for Multi-Task Learningcited by 16
  7. 2025Adversarial Robustness in Two-Stage Learning-to-Defer: Algorithms and Guaranteescited by 16
  8. 2025Mastering Multiple-Expert Routing: Realizable H-Consistency and Strong Guarantees for Learning to Defercited by 16
  9. 2024Optimal Query Allocation in Extractive QA with LLMs: A Learning-to-Defer Framework with Theoretical Guaranteescited by 15
  10. 2025Balancing the Scales: A Theoretical and Algorithmic Framework for Learning from Imbalanced Datacited by 15
  11. 2025Budgeted Multiple-Expert Deferralcited by 11
  12. 2025Improved Balanced Classification with Theoretically Grounded Loss Functionscited by 9
9 more
  1. 2025Why Ask One When You Can Ask k? Learning-to-Defer to the Top-k Expertscited by 6
  2. 2025Principled Algorithms for Optimizing Generalized Metrics in Binary Classificationcited by 6
  3. 2025Adversarial Robustness in One-Stage Learning-to-Defercited by 5
  4. 2026A Theoretical Framework for Modular Learning of Robust Generative Modelscited by 4
  5. 2026Optimized Deferral for Imbalanced Settingscited by 2
  6. 2026Linear-Core Surrogates: Smooth Loss Functions with Linear Rates for Classification and Structured Predictioncited by 2
  7. 2025Contrasting Cost-Agnostic and Cost-Sensitive Losses under Limited Model Capacity via mathcal H-consistencycited by 1
  8. 2026Online Learning-to-Defer with Varying Expertscited by 0
  9. 2026Principled Algorithms for Optimizing Generalized Metrics in Multi-Label Learningcited by 0