# Fast Nonlinear Vector Quantile Regression

arXiv: https://arxiv.org/abs/2205.14977 · HTML: https://mlsanity.com/paper/2205.14977/

## Key facts

- Cited by 12 papers in the map (data through 14 Oct 2026).
- Builds on 14 of its 41 references that are in the map.
- First posted on arXiv 30 May 2022.

## Builds on (most cited first, 14 of 14)

- [Sinkhorn Distances: Lightspeed Computation of Optimal Transportation Distances](https://mlsanity.com/paper/1306.0895/) (2013)
- [Computational Optimal Transport](https://mlsanity.com/paper/1803.00567/) (2018)
- [Stochastic First- and Zeroth-order Methods for Nonconvex Stochastic Programming](https://mlsanity.com/paper/1309.5549/) (2013)
- [CVXPY: A Python-Embedded Modeling Language for Convex Optimization](https://mlsanity.com/paper/1603.00943/) (2016)
- [Conformalized Quantile Regression](https://mlsanity.com/paper/1905.03222/) (2019)
- [PyOD: A Python Toolbox for Scalable Outlier Detection](https://mlsanity.com/paper/1901.01588/) (2019)
- [Stochastic Optimization for Large-scale Optimal Transport](https://mlsanity.com/paper/1605.08527/) (2016)
- [Kernel Operations on the GPU, with Autodiff, without Memory Overflows](https://mlsanity.com/paper/2004.11127/) (2020)
- [Do Neural Optimal Transport Solvers Work? A Continuous Wasserstein-2 Benchmark](https://mlsanity.com/paper/2106.01954/) (2021)
- [Conformal Prediction using Conditional Histograms](https://mlsanity.com/paper/2105.08747/) (2021)
- [Calibrated Multiple-Output Quantile Regression with Representation Learning](https://mlsanity.com/paper/2110.00816/) (2021)
- [Deep Non-Crossing Quantiles through the Partial Derivative](https://mlsanity.com/paper/2201.12848/) (2022)
- [Lecture Notes: Optimization for Machine Learning](https://mlsanity.com/paper/1909.03550/) (2019)
- [Causal inference for multiple continuous exposures via the multivariate generalized propensity score](https://mlsanity.com/paper/2008.13767/) (2020)

## Built on by (most cited first, 12 of 12)

- [How to Trust Your Diffusion Model: A Convex Optimization Approach to Conformal Risk Control](https://mlsanity.com/paper/2302.03791/) (2023)
- [Multivariate Conformal Prediction using Optimal Transport](https://mlsanity.com/paper/2502.03609/) (2025)
- [Optimal Transport-based Conformal Prediction](https://mlsanity.com/paper/2501.18991/) (2025)
- [Model-X Sequential Testing for Conditional Independence via Testing by Betting](https://mlsanity.com/paper/2210.00354/) (2022)
- [Adaptive Conditional Quantile Neural Processes](https://mlsanity.com/paper/2305.18777/) (2023)
- [Vector Quantile Regression on Manifolds](https://mlsanity.com/paper/2307.01037/) (2023)
- [Towards generalizable single-cell perturbation modeling via the Conditional Monge Gap](https://mlsanity.com/paper/2504.08328/) (2025)
- [Beyond Uncertainty Sets: Leveraging Optimal Transport to Extend Conformal Predictive Distributions to Multivariate Settings](https://mlsanity.com/paper/2511.15146/) (2025)
- [Neural Optimal Transport Meets Multivariate Conformal Prediction](https://mlsanity.com/paper/2509.25444/) (2025)
- [Minimax Multi-Target Conformal Prediction with Applications to Imaging Inverse Problems](https://mlsanity.com/paper/2511.13533/) (2025)
- [Brenier Isotonic Regression](https://mlsanity.com/paper/2603.10452/) (2026)
- [Two-way Guidance: Simultaneous Learning of Summary Statistics for Generative Bayes](https://mlsanity.com/paper/2610.12850/) (2026)

## Abstract (arXiv)

Quantile regression (QR) is a powerful tool for estimating one or more conditional quantiles of a target variable Y given explanatory features boldsymbolX. A limitation of QR is that it is only defined for scalar target variables, due to the formulation of its objective function, and since the notion of quantiles has no standard definition for multivariate distributions. Recently, vector quantile regression (VQR) was proposed as an extension of QR for vector-valued target variables, thanks to a meaningful generalization of the notion of quantiles to multivariate distributions via optimal transport. Despite its elegance, VQR is arguably not applicable in practice due to several limitations: (i) it assumes a linear model for the quantiles of the target boldsymbolY given the features boldsymbolX; (ii) its exact formulation is intractable even for modestly-sized problems in terms of target dimensions, number of regressed quantile levels, or number of features, and its relaxed dual formulation may violate the monotonicity of the estimated quantiles; (iii) no fast or scalable solvers for VQR currently exist. In this work we fully address these limitations, namely: (i) We extend VQR to the non-linear case, showing substantial improvement over linear VQR; (ii) We propose vector monotone rearrangement, a method which ensures the quantile functions estimated by VQR are monotone functions; (iii) We provide fast, GPU-accelerated solvers for linear and nonlinear VQR which maintain a fixed memory footprint, and demonstrate that they scale to millions of samples and thousands of quantile levels; (iv) We release an optimized python package of our solvers as to widespread the use of VQR in real-world applications.