# Generalized Linear Models with Structured Sparsity Estimators

arXiv: https://arxiv.org/abs/2104.14371 · HTML: https://mlsanity.com/paper/2104.14371/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 8 of its 29 references that are in the map.
- First posted on arXiv 29 Apr 2021.

## Builds on (most cited first, 8 of 8)

- [Robust Inference on Average Treatment Effects with Possibly More Covariates than Observations](https://mlsanity.com/paper/1309.4686/) (2013)
- [A General Theory of Hypothesis Tests and Confidence Regions for Sparse High Dimensional Models](https://mlsanity.com/paper/1412.8765/) (2014)
- [Post-Selection Inference for Generalized Linear Models with Many Controls](https://mlsanity.com/paper/1304.3969/) (2013)
- [The Benefit of Group Sparsity in Group Inference with De-biased Scaled Group Lasso](https://mlsanity.com/paper/1412.4170/) (2014)
- [Uniform Inference in High-dimensional Dynamic Panel Data Models](https://mlsanity.com/paper/1501.00478/) (2015)
- [Confidence regions for high-dimensional generalized linear models under sparsity](https://mlsanity.com/paper/1610.01353/) (2016)
- [Asymptotic Confidence Regions for High-dimensional Structured Sparsity](https://mlsanity.com/paper/1706.09231/) (2017)
- [Causal Inference by Quantile Regression Kink Designs](https://mlsanity.com/paper/1605.09773/) (2016)

## Abstract (arXiv)

In this paper, we introduce structured sparsity estimators in Generalized Linear Models. Structured sparsity estimators in the least squares loss are introduced by Stucky and van de Geer (2018) recently for fixed design and normal errors. We extend their results to debiased structured sparsity estimators with Generalized Linear Model based loss. Structured sparsity estimation means penalized loss functions with a possible sparsity structure used in the chosen norm. These include weighted group lasso, lasso and norms generated from convex cones. The significant difficulty is that it is not clear how to prove two oracle inequalities. The first one is for the initial penalized Generalized Linear Model estimator. Since it is not clear how a particular feasible-weighted nodewise regression may fit in an oracle inequality for penalized Generalized Linear Model, we need a second oracle inequality to get oracle bounds for the approximate inverse for the sample estimate of second-order partial derivative of Generalized Linear Model. Our contributions are fivefold: 1. We generalize the existing oracle inequality results in penalized Generalized Linear Models by proving the underlying conditions rather than assuming them. One of the key issues is the proof of a sample one-point margin condition and its use in an oracle inequality. 2. Our results cover even non sub-Gaussian errors and regressors. 3. We provide a feasible weighted nodewise regression proof which generalizes the results in the literature from a simple l_1 norm usage to norms generated from convex cones. 4. We realize that norms used in feasible nodewise regression proofs should be weaker or equal to the norms in penalized Generalized Linear Model loss. 5. We can debias the first step estimator via getting an approximate inverse of the singular-sample second order partial derivative of Generalized Linear Model loss.