# Confidence Sequences for Generalized Linear Models via Regret Analysis

arXiv: https://arxiv.org/abs/2504.16555 · HTML: https://mlsanity.com/paper/2504.16555/

## Key facts

- Cited by 10 papers in the map (data through 14 Oct 2026).
- Builds on 25 of its 70 references that are in the map.
- First posted on arXiv 23 Apr 2025.

## Builds on (most cited first, 25 of 25)

- [Laplace Redux -- Effortless Bayesian Deep Learning](https://mlsanity.com/paper/2106.14806/) (2021)
- [Estimating means of bounded random variables by betting](https://mlsanity.com/paper/2010.09686/) (2020)
- [Approximate Inference Turns Deep Networks into Gaussian Processes](https://mlsanity.com/paper/1906.01930/) (2019)
- [Scalable Generalized Linear Bandits: Online Computation and Hashing](https://mlsanity.com/paper/1706.00136/) (2017)
- [Relative Loss Bounds for On-line Density Estimation with the Exponential Family of Distributions](https://mlsanity.com/paper/1301.6677/) (2013)
- [Tight Concentrations and Confidence Sequences from the Regret of Universal Portfolio](https://mlsanity.com/paper/2110.14099/) (2021)
- [Logistic Regression: The Importance of Being Improper](https://mlsanity.com/paper/1803.09349/) (2018)
- [A Unified Confidence Sequence for Generalized Linear Models, with Applications to Bandits](https://mlsanity.com/paper/2407.13977/) (2024)
- [On Equivalence of Martingale Tail Bounds and Deterministic Regret Inequalities](https://mlsanity.com/paper/1510.03925/) (2015)
- [Improved Regret Bounds of (Multinomial) Logistic Bandits via Regret-to-Confidence-Set Conversion](https://mlsanity.com/paper/2310.18554/) (2023)
- [Online-to-PAC Conversions: Generalization Bounds via Regret Analysis](https://mlsanity.com/paper/2305.19674/) (2023)
- [Bregman Deviations of Generic Exponential Families](https://mlsanity.com/paper/2201.07306/) (2022)
- [Norm-Agnostic Linear Bandits](https://mlsanity.com/paper/2205.01257/) (2022)
- [Improved Algorithms for Stochastic Linear Bandits Using Tail Bounds for Martingale Mixtures](https://mlsanity.com/paper/2309.14298/) (2023)
- [FSP-Laplace: Function-Space Priors for the Laplace Approximation in Bayesian Deep Learning](https://mlsanity.com/paper/2407.13711/) (2024)
- [Likelihood Ratio Confidence Sets for Sequential Decision Making](https://mlsanity.com/paper/2311.04402/) (2023)
- [Tighter Confidence Bounds for Sequential Kernel Regression](https://mlsanity.com/paper/2403.12732/) (2024)
- [A Tight Excess Risk Bound via a Unified PAC-Bayesian-Rademacher-Shtarkov-MDL Complexity](https://mlsanity.com/paper/1710.07732/) (2017)
- [On the optimality of coin-betting for mean estimation](https://mlsanity.com/paper/2412.02640/) (2024)
- [Confidence Estimation via Sequential Likelihood Mixing](https://mlsanity.com/paper/2502.14689/) (2025)
- [Scale-invariant unconstrained online learning](https://mlsanity.com/paper/1708.07042/) (2017)
- [Refined Risk Bounds for Unbounded Losses via Transductive Priors](https://mlsanity.com/paper/2410.21621/) (2024)
- [Horizon-Independent Optimal Prediction with Log-Loss in Exponential Families](https://mlsanity.com/paper/1305.4324/) (2013)
- [Sequential Probability Assignment with Contexts: Minimax Regret, Contextual Shtarkov Sums, and Contextual Normalized Maximum Likelihood](https://mlsanity.com/paper/2410.03849/) (2024)
- [On the Minimax Regret of Sequential Probability Assignment via Square-Root Entropy](https://mlsanity.com/paper/2503.17823/) (2025)

## Built on by (most cited first, 10 of 10)

- [Generalized Linear Bandits: Almost Optimal Regret with One-Pass Update](https://mlsanity.com/paper/2507.11847/) (2025)
- [Eventually LIL Regret: Almost Sure lnln T Regret for a sub-Gaussian Mixture on Unbounded Data](https://mlsanity.com/paper/2512.12325/) (2025)
- [Sparse Optimistic Information Directed Sampling](https://mlsanity.com/paper/2510.24234/) (2025)
- [Principled Confidence Estimation for Deep Computed Tomography](https://mlsanity.com/paper/2602.05812/) (2026)
- [A Jointly Efficient and Optimal Algorithm for Heteroskedastic Generalized Linear Bandits with Adversarial Corruptions](https://mlsanity.com/paper/2602.10971/) (2026)
- [Self-Normalized Martingales and Uniform Regret Bounds for Linear Regression](https://mlsanity.com/paper/2605.01628/) (2026)
- [Best Arm Identification in Generalized Linear Bandits via Hybrid Feedback](https://mlsanity.com/paper/2605.05745/) (2026)
- [Generalized Linear Bandits with Memory](https://mlsanity.com/paper/2608.15848/) (2026)
- [Linear Bandits with Non-i.i.d. Noise](https://mlsanity.com/paper/2505.20017/) (2025)
- [Beyond Modern Asymptotics for Log-Likelihood Ratios in Logistic Regression](https://mlsanity.com/paper/2608.02507/) (2026)

## Abstract (arXiv)

We develop a methodology for constructing confidence sets for parameters of statistical models via a reduction to sequential prediction. Our key observation is that for any generalized linear model (GLM), one can construct an associated game of sequential probability assignment such that achieving low regret in the game implies a high-probability upper bound on the excess likelihood of the true parameter of the GLM. This allows us to develop a scheme that we call online-to-confidence-set conversions, which effectively reduces the problem of proving the desired statistical claim to an algorithmic question. We study two varieties of this conversion scheme: 1) analytical conversions that only require proving the existence of algorithms with low regret and provide confidence sets centered at the maximum-likelihood estimator 2) algorithmic conversions that actively leverage the output of the online algorithm to construct confidence sets (and may be centered at other, adaptively constructed point estimators). The resulting methodology recovers all state-of-the-art confidence set constructions within a single framework, and also provides several new types of confidence sets that were previously unknown in the literature.