# Adversarial Regression with Doubly Non-negative Weighting Matrices

arXiv: https://arxiv.org/abs/2109.14875 · HTML: https://mlsanity.com/paper/2109.14875/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- Builds on 21 of its 48 references that are in the map.
- First posted on arXiv 30 Sep 2021.

## Builds on (most cited first, 21 of 21)

- ["Why Should I Trust You?": Explaining the Predictions of Any Classifier](https://mlsanity.com/paper/1602.04938/) (2016)
- [Equality of Opportunity in Supervised Learning](https://mlsanity.com/paper/1610.02413/) (2016)
- [Know What You Don't Know: Unanswerable Questions for SQuAD](https://mlsanity.com/paper/1806.03822/) (2018)
- [Learning to Reweight Examples for Robust Deep Learning](https://mlsanity.com/paper/1803.09050/) (2018)
- [Deep Speech 2: End-to-End Speech Recognition in English and Mandarin](https://mlsanity.com/paper/1512.02595/) (2015)
- [Interpretation of Neural Networks is Fragile](https://mlsanity.com/paper/1710.10547/) (2017)
- [Fairness Without Demographics in Repeated Loss Minimization](https://mlsanity.com/paper/1806.08010/) (2018)
- [On the Robustness of Interpretability Methods](https://mlsanity.com/paper/1806.08049/) (2018)
- [Fairness without Demographics through Adversarially Reweighted Learning](https://mlsanity.com/paper/2006.13114/) (2020)
- [Variance-based regularization with convex objectives](https://mlsanity.com/paper/1610.02581/) (2016)
- [Robust Optimization for Fairness with Noisy Protected Groups](https://mlsanity.com/paper/2002.09343/) (2020)
- [Coping with Label Shift via Distributionally Robust Optimisation](https://mlsanity.com/paper/2010.12230/) (2020)
- [From Parity to Preference-based Notions of Fairness in Classification](https://mlsanity.com/paper/1707.00010/) (2017)
- [Does Distributionally Robust Supervised Learning Give Robust Classifiers?](https://mlsanity.com/paper/1611.02041/) (2016)
- [Residuals-based distributionally robust optimization with covariate information](https://mlsanity.com/paper/2012.01088/) (2020)
- [Distributionally robust stochastic programs with side information based on trimmings -- Extended version](https://mlsanity.com/paper/2009.10592/) (2020)
- [Distributionally Robust Local Non-parametric Conditional Estimation](https://mlsanity.com/paper/2010.05373/) (2020)
- [Robustifying Conditional Portfolio Decisions via Optimal Transport](https://mlsanity.com/paper/2103.16451/) (2021)
- [Bridging Bayesian and Minimax Mean Square Error Estimation via Wasserstein Distributionally Robust Optimization](https://mlsanity.com/paper/1911.03539/) (2019)
- [Robust Nonparametric Regression under Huber's ε-contamination Model](https://mlsanity.com/paper/1805.10406/) (2018)
- [Density Estimation with Contaminated Data: Minimax Rates and Theory of Adaptation](https://mlsanity.com/paper/1712.07801/) (2017)

## Built on by (most cited first, 5 of 5)

- [Sobolev Transport: A Scalable Metric for Probability Measures with Graph Metrics](https://mlsanity.com/paper/2202.10723/) (2022)
- [ProKeR: A Kernel Perspective on Few-Shot Adaptation of Large Vision-Language Models](https://mlsanity.com/paper/2501.11175/) (2025)
- [Scalable Unbalanced Sobolev Transport for Measures on a Graph](https://mlsanity.com/paper/2302.12498/) (2023)
- [Generalized Sobolev Transport for Probability Measures on a Graph](https://mlsanity.com/paper/2402.04516/) (2024)
- [Optimal Transport for Measures with Noisy Tree Metric](https://mlsanity.com/paper/2310.13653/) (2023)

## Abstract (arXiv)

Many machine learning tasks that involve predicting an output response can be solved by training a weighted regression model. Unfortunately, the predictive power of this type of models may severely deteriorate under low sample sizes or under covariate perturbations. Reweighting the training samples has aroused as an effective mitigation strategy to these problems. In this paper, we propose a novel and coherent scheme for kernel-reweighted regression by reparametrizing the sample weights using a doubly non-negative matrix. When the weighting matrix is confined in an uncertainty set using either the log-determinant divergence or the Bures-Wasserstein distance, we show that the adversarially reweighted estimate can be solved efficiently using first-order methods. Numerical experiments show that our reweighting strategy delivers promising results on numerous datasets.