# Learning Conditional Expectation Operators via Functional Newton Updates

arXiv: https://arxiv.org/abs/2609.35598 · HTML: https://mlsanity.com/paper/2609.35598/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 19 of its 42 references that are in the map.
- First posted on arXiv 28 Sep 2026.

## Builds on (most cited first, 19 of 19)

- [XGBoost: A Scalable Tree Boosting System](https://mlsanity.com/paper/1603.02754/) (2016)
- [Kernel Mean Embedding of Distributions: A Review and Beyond](https://mlsanity.com/paper/1605.09522/) (2016)
- [Why do tree-based models still outperform deep learning on tabular data?](https://mlsanity.com/paper/2207.08815/) (2022)
- [VAMPnets: Deep learning of molecular kinetics](https://mlsanity.com/paper/1710.06012/) (2017)
- [Learning Dynamical Systems via Koopman Operator Regression in Reproducing Kernel Hilbert Spaces](https://mlsanity.com/paper/2205.14027/) (2022)
- [Making Linear MDPs Practical via Contrastive Representation Learning](https://mlsanity.com/paper/2207.07150/) (2022)
- [Sharp Spectral Rates for Koopman Operator Learning](https://mlsanity.com/paper/2302.02004/) (2023)
- [High-Dimensional Density Ratio Estimation with Extensions to Approximate Likelihood Computation](https://mlsanity.com/paper/1404.7063/) (2014)
- [Gradient and Newton Boosting for Classification and Regression](https://mlsanity.com/paper/1808.03064/) (2018)
- [Neural Conditional Probability for Uncertainty Quantification](https://mlsanity.com/paper/2407.01171/) (2024)
- [Spectral Representation Learning for Conditional Moment Models](https://mlsanity.com/paper/2210.16525/) (2022)
- [Spectral Representation for Causal Estimation with Hidden Confounders](https://mlsanity.com/paper/2407.10448/) (2024)
- [Randomized Gradient Boosting Machine](https://mlsanity.com/paper/1810.10158/) (2018)
- [Operator World Models for Reinforcement Learning](https://mlsanity.com/paper/2406.19861/) (2024)
- [Self-Supervised Evolution Operator Learning for High-Dimensional Dynamical Systems](https://mlsanity.com/paper/2505.18671/) (2025)
- [Non-linear Canonical Correlation Analysis: A Compressed Representation Approach](https://mlsanity.com/paper/1810.13259/) (2018)
- [Representation Learning for Equivariant Inference with Guarantees](https://mlsanity.com/paper/2505.19809/) (2025)
- [Outcome-Aware Spectral Feature Learning for Instrumental Variable Regression](https://mlsanity.com/paper/2512.00919/) (2025)
- [Toward Scalable and Valid Conditional Independence Testing with Spectral Representations](https://mlsanity.com/paper/2512.19510/) (2025)

## Abstract (arXiv)

We introduce the Functional Spectral-Newton Method (FSNM) for learning the leading singular structure of a conditional expectation operator without fixing a basis or reproducing kernel Hilbert space. FSNM fits a low-rank representation of the centered joint-to-product density ratio kernel by alternating functional Newton updates. Each update reduces to a preconditioned regression, which we approximate with vector-valued regression trees in a stagewise boosting procedure. At the population level, we establish descent and an O(1/T) best-iterate block-stationarity rate under a relative weak-learner accuracy condition, and show that every nondegenerate local minimum over the full centered L2 spaces is a globally optimal rank-d approximation. Synthetic experiments show that FSNM recovers a low-rank density ratio and its leading spectral structure, and that the same learned kernel can answer multiple conditional queries without refitting.