# Hard Shape-Constrained Kernel Machines

arXiv: https://arxiv.org/abs/2005.12636 · HTML: https://mlsanity.com/paper/2005.12636/

## Key facts

- Cited by 13 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 26 May 2020.

## Built on by (most cited first, 13 of 13)

- [On Controller Tuning with Time-Varying Bayesian Optimization](https://mlsanity.com/paper/2207.11120/) (2022)
- [Linear Partial Monitoring for Sequential Decision-Making: Algorithms, Regret Bounds and Applications](https://mlsanity.com/paper/2302.03683/) (2023)
- [Handling Hard Affine SDP Shape Constraints in RKHSs](https://mlsanity.com/paper/2101.01519/) (2021)
- [Capturing and incorporating expert knowledge into machine learning models for quality prediction in manufacturing](https://mlsanity.com/paper/2202.02003/) (2022)
- [Approximation of optimization problems with constraints through kernel Sum-Of-Squares](https://mlsanity.com/paper/2301.06339/) (2023)
- [Kernel Conditional Moment Constraints for Confounding Robust Inference](https://mlsanity.com/paper/2302.13348/) (2023)
- [A Specialized Semismooth Newton Method for Kernel-Based Optimal Transport](https://mlsanity.com/paper/2310.14087/) (2023)
- [Exploiting Concavity Information in Gaussian Process Contextual Bandit Optimization](https://mlsanity.com/paper/2503.10836/) (2025)
- [Concentration inequalities for semidefinite least squares based on data](https://mlsanity.com/paper/2509.13166/) (2025)
- [A Convex Framework for Confounding Robust Inference](https://mlsanity.com/paper/2309.12450/) (2023)
- [Output-Constrained Decision Trees](https://mlsanity.com/paper/2405.15314/) (2024)
- [DiscoverDCP: A Data-Driven Approach for Construction of Disciplined Convex Programs via Symbolic Regression](https://mlsanity.com/paper/2512.15721/) (2025)
- [Kernel-Based Learning of Safety Barriers](https://mlsanity.com/paper/2601.12002/) (2026)

## Abstract (arXiv)

Shape constraints (such as non-negativity, monotonicity, convexity) play a central role in a large number of applications, as they usually improve performance for small sample size and help interpretability. However enforcing these shape requirements in a hard fashion is an extremely challenging problem. Classically, this task is tackled (i) in a soft way (without out-of-sample guarantees), (ii) by specialized transformation of the variables on a case-by-case basis, or (iii) by using highly restricted function classes, such as polynomials or polynomial splines. In this paper, we prove that hard affine shape constraints on function derivatives can be encoded in kernel machines which represent one of the most flexible and powerful tools in machine learning and statistics. Particularly, we present a tightened second-order cone constrained reformulation, that can be readily implemented in convex solvers. We prove performance guarantees on the solution, and demonstrate the efficiency of the approach in joint quantile regression with applications to economics and to the analysis of aircraft trajectories, among others.