# Stochastic Gradients under Nuisances

arXiv: https://arxiv.org/abs/2508.20326 · HTML: https://mlsanity.com/paper/2508.20326/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 28 Aug 2025.

## Built on by (most cited first, 1 of 1)

- [DoubleGen: Debiased Generative Modeling of Counterfactuals](https://mlsanity.com/paper/2509.16842/) (2025)

## Abstract (arXiv)

Stochastic gradient optimization is the dominant learning paradigm for a variety of scenarios, from classical supervised learning to modern self-supervised learning. We consider stochastic gradient algorithms for learning problems whose objectives rely on unknown nuisance parameters, and establish non-asymptotic convergence guarantees. Our results show that, while the presence of a nuisance can alter the optimum and upset the optimization trajectory, the classical stochastic gradient algorithm may still converge under appropriate conditions, such as Neyman orthogonality. Moreover, even when Neyman orthogonality is not satisfied, we show that an algorithm variant with approximately orthogonalized updates (with an approximately orthogonalized gradient oracle) may achieve similar convergence rates. Examples from orthogonal statistical learning/double machine learning and causal inference are discussed.