# Confidence Sets under Generalized Self-Concordance

arXiv: https://arxiv.org/abs/2301.00260 · HTML: https://mlsanity.com/paper/2301.00260/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 15 of its 48 references that are in the map.
- First posted on arXiv 31 Dec 2022.

## Builds on (most cited first, 15 of 15)

- [Limitations of the Empirical Fisher Approximation for Natural Gradient Descent](https://mlsanity.com/paper/1905.12558/) (2019)
- [Generalization Properties of Learning with Random Features](https://mlsanity.com/paper/1602.04474/) (2016)
- [Improved Optimistic Algorithms for Logistic Bandits](https://mlsanity.com/paper/2002.07530/) (2020)
- [Gone Fishing: Neural Active Learning with Fisher Embeddings](https://mlsanity.com/paper/2106.09675/) (2021)
- [The Landscape of Empirical Risk for Non-convex Losses](https://mlsanity.com/paper/1607.06534/) (2016)
- [Generalized Self-Concordant Functions: A Recipe for Newton-Type Methods](https://mlsanity.com/paper/1703.04599/) (2017)
- [Finite-sample analysis of M-estimators using self-concordance](https://mlsanity.com/paper/1810.06838/) (2018)
- [Beyond Least-Squares: Fast Rates for Regularized Empirical Risk Minimization through Self-Concordance](https://mlsanity.com/paper/1902.03046/) (2019)
- [On the Variance of the Fisher Information for Deep Learning](https://mlsanity.com/paper/2107.04205/) (2021)
- [Robust Inference via Multiplier Bootstrap](https://mlsanity.com/paper/1903.07208/) (2019)
- [A New Perspective on Robust M-Estimation: Finite Sample Theory and Applications to Dependence-Adjusted Multiple Testing](https://mlsanity.com/paper/1711.05381/) (2017)
- [Simultaneous Inference for Pairwise Graphical Models with Generalized Score Matching](https://mlsanity.com/paper/1905.06261/) (2019)
- [Nonuniformity of P-values Can Occur Early in Diverging Dimensions](https://mlsanity.com/paper/1705.03604/) (2017)
- [Orthogonal Statistical Learning with Self-Concordant Loss](https://mlsanity.com/paper/2205.00350/) (2022)
- [Black-box optimization with a politician](https://mlsanity.com/paper/1602.04847/) (2016)

## Built on by (most cited first, 1 of 1)

- [Fast Rates for Inverse Reinforcement Learning](https://mlsanity.com/paper/2605.14599/) (2026)

## Abstract (arXiv)

This paper revisits a fundamental problem in statistical inference from a non-asymptotic theoretical viewpoint unicodex2013 the construction of confidence sets. We establish a finite-sample bound for the estimator, characterizing its asymptotic behavior in a non-asymptotic fashion. An important feature of our bound is that its dimension dependency is captured by the effective dimension unicodex2013 the trace of the limiting sandwich covariance unicodex2013 which can be much smaller than the parameter dimension in some regimes. We then illustrate how the bound can be used to obtain a confidence set whose shape is adapted to the optimization landscape induced by the loss function. Unlike previous works that rely heavily on the strong convexity of the loss function, we only assume the Hessian is lower bounded at optimum and allow it to gradually becomes degenerate. This property is formalized by the notion of generalized self-concordance which originated from convex optimization. Moreover, we demonstrate how the effective dimension can be estimated from data and characterize its estimation accuracy. We apply our results to maximum likelihood estimation with generalized linear models, score matching with exponential families, and hypothesis testing with Rao's score test.