# Inference with non-differentiable surrogate loss in a general high-dimensional classification framework

arXiv: https://arxiv.org/abs/2405.11723 · HTML: https://mlsanity.com/paper/2405.11723/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 16 of its 47 references that are in the map.
- First posted on arXiv 20 May 2024.

## Builds on (most cited first, 16 of 16)

- [High-Dimensional Asymptotics of Prediction: Ridge Regression and Classification](https://mlsanity.com/paper/1507.03003/) (2015)
- [Double/Debiased/Neyman Machine Learning of Treatment Effects](https://mlsanity.com/paper/1701.08687/) (2017)
- [Residual Weighted Learning for Estimating Individualized Treatment Rules](https://mlsanity.com/paper/1508.03179/) (2015)
- [A General Theory of Hypothesis Tests and Confidence Regions for Sparse High Dimensional Models](https://mlsanity.com/paper/1412.8765/) (2014)
- [Distributed Inference for Linear Support Vector Machine](https://mlsanity.com/paper/1811.11922/) (2018)
- [Semi-supervised Triply Robust Inductive Transfer Learning](https://mlsanity.com/paper/2209.04977/) (2022)
- [Global and Simultaneous Hypothesis Testing for High-Dimensional Logistic Regression Models](https://mlsanity.com/paper/1805.06970/) (2018)
- [Optimal and Safe Estimation for High-Dimensional Semi-Supervised Learning](https://mlsanity.com/paper/2011.14185/) (2020)
- [High-dimensional simultaneous inference with the bootstrap](https://mlsanity.com/paper/1606.03940/) (2016)
- [Efficient augmentation and relaxation learning for individualized treatment rules using observational data](https://mlsanity.com/paper/1901.00663/) (2019)
- [Consistency of semi-supervised learning algorithms on graphs: Probit and one-hot methods](https://mlsanity.com/paper/1906.07658/) (2019)
- [Multicategory Angle-based Learning for Estimating Optimal Dynamic Treatment Regimes with Censored Data](https://mlsanity.com/paper/2001.04629/) (2020)
- [Estimation and inference on high-dimensional individualized treatment rule in observational data using split-and-pooled de-correlated score](https://mlsanity.com/paper/2007.04445/) (2020)
- [Bayesian Nonparametric Common Atoms Regression for Generating Synthetic Controls in Clinical Trials](https://mlsanity.com/paper/2201.00068/) (2021)
- [Optimal Discriminant Analysis in High-Dimensional Latent Factor Models](https://mlsanity.com/paper/2210.12862/) (2022)
- [Model-Assisted Uniformly Honest Inference for Optimal Treatment Regimes in High Dimension](https://mlsanity.com/paper/2105.04099/) (2021)

## Abstract (arXiv)

Penalized empirical risk minimization with a surrogate loss function is often used to learn a high-dimensional linear decision rule in classification problems. Although much of the literature focus on the generalization error, there is a lack of inference procedures for identifying the driving factors of the estimated decision rule, especially when the surrogate loss is non-differentiable. We propose a kernel-smoothed decorrelated score to construct hypothesis tests and interval estimators for a linear decision rule estimated using a piece-wise linear surrogate loss, which has a discontinuous gradient and non-regular Hessian. Specifically, we adopt kernel approximations to smooth the discontinuous gradient near discontinuity points and approximate the non-regular Hessian of the surrogate loss. In applications where additional nuisance parameters are involved, we propose a novel cross-fitted version to accommodate flexible nuisance estimates and kernel approximations. We establish the limiting distribution of the kernel-smoothed decorrelated score and its cross-fitted version in a high-dimensional setup. Simulation and real data analysis are conducted to demonstrate the validity and the superiority of the proposed method.