mlsanityData to 7 Oct 2026

Differentiable Spline Approximations

arXiv 2110.01532 ↗First posted 4 Oct 2021cs.LG stat.MLcited by 4

Not on the map. Why?

Abstract (arXiv)

The paradigm of differentiable programming has significantly enhanced the scope of machine learning via the judicious use of gradient-based optimization. However, standard differentiable programming methods (such as autodiff) typically require that the machine learning models be differentiable, limiting their applicability. Our goal in this paper is to use a new, principled approach to extend gradient-based optimization to functions well modeled by splines, which encompass a large family of piecewise polynomial models. We derive the form of the (weak) Jacobian of such functions and show that it exhibits a block-sparse structure that can be computed implicitly and efficiently. Overall, we show that leveraging this redesigned Jacobian in the form of a differentiable "layer" in predictive models leads to improved performance in diverse applications such as image segmentation, 3D point cloud reconstruction, and finite element analysis.

Key facts. Cited by 4 papers in the map (data through 7 Oct 2026). Builds on 28 of its 66 references that are in the map. First posted on arXiv 4 Oct 2021.

Builds on

28 of 66 references.

References · 28

  1. 2015U-Net: Convolutional Networks for Biomedical Image Segmentationcited by 19,604
  2. 2020Learning to Simulate Complex Physics with Graph Networkscited by 867
  3. 2015Automatic differentiation in machine learning: a surveycited by 846
  4. 2017OptNet: Differentiable Optimization as a Layer in Neural Networkscited by 588
  5. 2021Physics-informed neural networks (PINNs) for fluid mechanics: A reviewcited by 470
  6. 2019Differentiable Convex Optimization Layerscited by 458
  7. 2019DiffTaichi: Differentiable Programming for Physical Simulationcited by 248
  8. 2020hp-VPINNs: Variational Physics-Informed Neural Networks With Domain Decompositioncited by 236
  9. 2019Understanding Straight-Through Estimator in Training Activation Quantized Neural Netscited by 225
  10. 2019Differentiation of Blackbox Combinatorial Solverscited by 224
  11. 2019Learning to Predict 3D Objects with an Interpolation-based Differentiable Renderercited by 221
  12. 2017Hidden Physics Models: Machine Learning of Nonlinear Partial Differential Equationscited by 208
16 more
  1. 2020Fast Differentiable Sorting and Rankingcited by 196
  2. 2019CvxNet: Learnable Convex Decompositioncited by 173
  3. 2017SplineCNN: Fast Geometric Deep Learning with Continuous B-Spline Kernelscited by 160
  4. 2020Understanding and mitigating gradient pathologies in physics-informed neural networkscited by 134
  5. 2020Learning to Control PDEs with Differentiable Physicscited by 131
  6. 2019Differentiable Ranks and Sorting using Optimal Transportcited by 113
  7. 2016A Differentiable Physics Engine for Deep Learning in Roboticscited by 107
  8. 2020ParSeNet: A Parametric Surface Fitting Network for 3D Point Cloudscited by 96
  9. 2019A Differentiable Programming System to Bridge Machine Learning and Scientific Computingcited by 81
  10. 2020Learning with Differentiable Perturbed Optimizerscited by 81
  11. 2020Scalable Differentiable Physics for Learning and Controlcited by 72
  12. 2018Differentiable Dynamic Programming for Structured Prediction and Attentioncited by 58
  13. 2013Estimating or Propagating Gradients Through Stochastic Neuronscited by 54
  14. 2020Differentiable Top-k Operator with Optimal Transportcited by 17
  15. 2019Accelerating PDE-constrained Inverse Solutions with Deep Learning and Reduced Order Modelscited by 6
  16. 2020A Differentiable Ranking Metric Using Relaxed Sorting Operation for Top-K Recommender Systemscited by 1

Built on by

Cited by year

Citing papers per year: 2023: 1; 2024: 0; 2025: 3.

4 papers.

  1. 2023Differentiable Visual Computing for Inverse Problems and Machine Learningcited by 7
  2. 2025Physics-Informed Deep B-Spline Networkscited by 1
  3. 2025Neural Spline Operators for Risk Quantification in Stochastic Systemscited by 1
  4. 2025Graph-based Point Cloud Surface Reconstruction using B-Splinescited by 0