# Non-parametric Models for Non-negative Functions

arXiv: https://arxiv.org/abs/2007.03926 · HTML: https://mlsanity.com/paper/2007.03926/

## Key facts

- Cited by 41 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 8 Jul 2020.

## Built on by (most cited first, 30 of 41)

- [Subtractive Mixture Models via Squaring: Representation and Learning](https://mlsanity.com/paper/2310.00724/) (2023)
- [PSD Representations for Effective Probability Models](https://mlsanity.com/paper/2106.16116/) (2021)
- [Sum of Squares Circuits](https://mlsanity.com/paper/2408.11778/) (2024)
- [On the Relationship Between Monotone and Squared Probabilistic Circuits](https://mlsanity.com/paper/2408.00876/) (2024)
- [Near-optimal estimation of smooth transport maps with kernel sums-of-squares](https://mlsanity.com/paper/2112.01907/) (2021)
- [Squared Neural Families: A New Class of Tractable Density Models](https://mlsanity.com/paper/2305.13552/) (2023)
- [Sampling from Arbitrary Functions via PSD Models](https://mlsanity.com/paper/2110.10527/) (2021)
- [UTOPIA: Universally Trainable Optimal Prediction Intervals Aggregation](https://mlsanity.com/paper/2306.16549/) (2023)
- [Exact, Fast and Expressive Poisson Point Processes via Squared Neural Families](https://mlsanity.com/paper/2402.09608/) (2024)
- [Learning PSD-valued functions using kernel sums-of-squares](https://mlsanity.com/paper/2111.11306/) (2021)
- [Estimation Beyond Data Reweighting: Kernel Method of Moments](https://mlsanity.com/paper/2305.10898/) (2023)
- [How to Square Tensor Networks and Circuits Without Squaring Them](https://mlsanity.com/paper/2512.17090/) (2025)
- [Handling Hard Affine SDP Shape Constraints in RKHSs](https://mlsanity.com/paper/2101.01519/) (2021)
- [High-dimensional additive Gaussian processes under monotonicity constraints](https://mlsanity.com/paper/2205.08528/) (2022)
- [Non-Parametric Learning of Stochastic Differential Equations with Non-asymptotic Fast Rates of Convergence](https://mlsanity.com/paper/2305.15557/) (2023)
- [Kernel Sum of Squares for Data Adapted Kernel Learning of Dynamical Systems from Data: A global optimization approach](https://mlsanity.com/paper/2408.06465/) (2024)
- [Generalization Bounds of Surrogate Policies for Combinatorial Optimization Problems](https://mlsanity.com/paper/2407.17200/) (2024)
- [Universal Prediction Band via Semi-Definite Programming](https://mlsanity.com/paper/2103.17203/) (2021)
- [A Note on Optimizing Distributions using Kernel Mean Embeddings](https://mlsanity.com/paper/2106.09994/) (2021)
- [Adaptive joint distribution learning](https://mlsanity.com/paper/2110.04829/) (2021)
- [Non-Convex Optimization with Certificates and Fast Rates Through Kernel Sums of Squares](https://mlsanity.com/paper/2204.04970/) (2022)
- [Approximation of optimization problems with constraints through kernel Sum-Of-Squares](https://mlsanity.com/paper/2301.06339/) (2023)
- [Sobolev Space Regularised Pre Density Models](https://mlsanity.com/paper/2307.13763/) (2023)
- [Kernelized Normalizing Constant Estimation: Bridging Bayesian Quadrature and Bayesian Optimization](https://mlsanity.com/paper/2401.05716/) (2024)
- [Regularized f-Divergence Kernel Tests](https://mlsanity.com/paper/2601.19755/) (2026)
- [Efficient Sampling of Stochastic Differential Equations with Positive Semi-Definite Models](https://mlsanity.com/paper/2303.17109/) (2023)
- [Sequential transport maps using SoS density estimation and α-divergences](https://mlsanity.com/paper/2402.17943/) (2024)
- [Squared families: Searching beyond regular probability models](https://mlsanity.com/paper/2503.21128/) (2025)
- [Scalable and adaptive prediction bands with kernel sum-of-squares](https://mlsanity.com/paper/2505.21039/) (2025)
- [How to Approximate Inference with Subtractive Mixture Models](https://mlsanity.com/paper/2604.16714/) (2026)

## Abstract (arXiv)

Linear models have shown great effectiveness and flexibility in many fields such as machine learning, signal processing and statistics. They can represent rich spaces of functions while preserving the convexity of the optimization problems where they are used, and are simple to evaluate, differentiate and integrate. However, for modeling non-negative functions, which are crucial for unsupervised learning, density estimation, or non-parametric Bayesian methods, linear models are not applicable directly. Moreover, current state-of-the-art models like generalized linear models either lead to non-convex optimization problems, or cannot be easily integrated. In this paper we provide the first model for non-negative functions which benefits from the same good properties of linear models. In particular, we prove that it admits a representer theorem and provide an efficient dual formulation for convex problems. We study its representation power, showing that the resulting space of functions is strictly richer than that of generalized linear models. Finally we extend the model and the theoretical results to functions with outputs in convex cones. The paper is complemented by an experimental evaluation of the model showing its effectiveness in terms of formulation, algorithmic derivation and practical results on the problems of density estimation, regression with heteroscedastic errors, and multiple quantile regression.