# Differentiable Spline Approximations

arXiv: https://arxiv.org/abs/2110.01532 · HTML: https://mlsanity.com/paper/2110.01532/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- Builds on 28 of its 66 references that are in the map.
- First posted on arXiv 4 Oct 2021.

## Builds on (most cited first, 28 of 28)

- [U-Net: Convolutional Networks for Biomedical Image Segmentation](https://mlsanity.com/paper/1505.04597/) (2015)
- [Learning to Simulate Complex Physics with Graph Networks](https://mlsanity.com/paper/2002.09405/) (2020)
- [Automatic differentiation in machine learning: a survey](https://mlsanity.com/paper/1502.05767/) (2015)
- [OptNet: Differentiable Optimization as a Layer in Neural Networks](https://mlsanity.com/paper/1703.00443/) (2017)
- [Physics-informed neural networks (PINNs) for fluid mechanics: A review](https://mlsanity.com/paper/2105.09506/) (2021)
- [Differentiable Convex Optimization Layers](https://mlsanity.com/paper/1910.12430/) (2019)
- [DiffTaichi: Differentiable Programming for Physical Simulation](https://mlsanity.com/paper/1910.00935/) (2019)
- [hp-VPINNs: Variational Physics-Informed Neural Networks With Domain Decomposition](https://mlsanity.com/paper/2003.05385/) (2020)
- [Understanding Straight-Through Estimator in Training Activation Quantized Neural Nets](https://mlsanity.com/paper/1903.05662/) (2019)
- [Differentiation of Blackbox Combinatorial Solvers](https://mlsanity.com/paper/1912.02175/) (2019)
- [Learning to Predict 3D Objects with an Interpolation-based Differentiable Renderer](https://mlsanity.com/paper/1908.01210/) (2019)
- [Hidden Physics Models: Machine Learning of Nonlinear Partial Differential Equations](https://mlsanity.com/paper/1708.00588/) (2017)
- [Fast Differentiable Sorting and Ranking](https://mlsanity.com/paper/2002.08871/) (2020)
- [CvxNet: Learnable Convex Decomposition](https://mlsanity.com/paper/1909.05736/) (2019)
- [SplineCNN: Fast Geometric Deep Learning with Continuous B-Spline Kernels](https://mlsanity.com/paper/1711.08920/) (2017)
- [Understanding and mitigating gradient pathologies in physics-informed neural networks](https://mlsanity.com/paper/2001.04536/) (2020)
- [Learning to Control PDEs with Differentiable Physics](https://mlsanity.com/paper/2001.07457/) (2020)
- [Differentiable Ranks and Sorting using Optimal Transport](https://mlsanity.com/paper/1905.11885/) (2019)
- [A Differentiable Physics Engine for Deep Learning in Robotics](https://mlsanity.com/paper/1611.01652/) (2016)
- [ParSeNet: A Parametric Surface Fitting Network for 3D Point Clouds](https://mlsanity.com/paper/2003.12181/) (2020)
- [A Differentiable Programming System to Bridge Machine Learning and Scientific Computing](https://mlsanity.com/paper/1907.07587/) (2019)
- [Learning with Differentiable Perturbed Optimizers](https://mlsanity.com/paper/2002.08676/) (2020)
- [Scalable Differentiable Physics for Learning and Control](https://mlsanity.com/paper/2007.02168/) (2020)
- [Differentiable Dynamic Programming for Structured Prediction and Attention](https://mlsanity.com/paper/1802.03676/) (2018)
- [Estimating or Propagating Gradients Through Stochastic Neurons](https://mlsanity.com/paper/1305.2982/) (2013)
- [Differentiable Top-k Operator with Optimal Transport](https://mlsanity.com/paper/2002.06504/) (2020)
- [Accelerating PDE-constrained Inverse Solutions with Deep Learning and Reduced Order Models](https://mlsanity.com/paper/1912.08864/) (2019)
- [A Differentiable Ranking Metric Using Relaxed Sorting Operation for Top-K Recommender Systems](https://mlsanity.com/paper/2008.13141/) (2020)

## Built on by (most cited first, 4 of 4)

- [Differentiable Visual Computing for Inverse Problems and Machine Learning](https://mlsanity.com/paper/2312.04574/) (2023)
- [Physics-Informed Deep B-Spline Networks](https://mlsanity.com/paper/2503.16777/) (2025)
- [Neural Spline Operators for Risk Quantification in Stochastic Systems](https://mlsanity.com/paper/2508.20288/) (2025)
- [Graph-based Point Cloud Surface Reconstruction using B-Splines](https://mlsanity.com/paper/2509.16050/) (2025)

## Abstract (arXiv)

The paradigm of differentiable programming has significantly enhanced the scope of machine learning via the judicious use of gradient-based optimization. However, standard differentiable programming methods (such as autodiff) typically require that the machine learning models be differentiable, limiting their applicability. Our goal in this paper is to use a new, principled approach to extend gradient-based optimization to functions well modeled by splines, which encompass a large family of piecewise polynomial models. We derive the form of the (weak) Jacobian of such functions and show that it exhibits a block-sparse structure that can be computed implicitly and efficiently. Overall, we show that leveraging this redesigned Jacobian in the form of a differentiable "layer" in predictive models leads to improved performance in diverse applications such as image segmentation, 3D point cloud reconstruction, and finite element analysis.