# lapanda: A Matrix-Free Differentiable Solver for Nonconvex Constrained Optimization Layers

arXiv: https://arxiv.org/abs/2609.33630 · HTML: https://mlsanity.com/paper/2609.33630/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 22 of its 27 references that are in the map.
- First posted on arXiv 27 Sep 2026.

## Builds on (most cited first, 22 of 22)

- [OptNet: Differentiable Optimization as a Layer in Neural Networks](https://mlsanity.com/paper/1703.00443/) (2017)
- [Differentiable Convex Optimization Layers](https://mlsanity.com/paper/1910.12430/) (2019)
- [Efficient and Modular Implicit Differentiation](https://mlsanity.com/paper/2105.15183/) (2021)
- [Differentiable MPC for End-to-end Planning and Control](https://mlsanity.com/paper/1810.13400/) (2018)
- [End-to-End Constrained Optimization Learning: A Survey](https://mlsanity.com/paper/2103.16378/) (2021)
- [Differentiating Through a Cone Program](https://mlsanity.com/paper/1904.09043/) (2019)
- [Differentiable Integrated Motion Prediction and Planning with Learnable Cost Function for Autonomous Driving](https://mlsanity.com/paper/2207.10422/) (2022)
- [acados: a modular open-source framework for fast embedded optimal control](https://mlsanity.com/paper/1910.13753/) (2019)
- [Theseus: A Library for Differentiable Nonlinear Optimization](https://mlsanity.com/paper/2207.09442/) (2022)
- [DiffStack: A Differentiable and Modular Control Stack for Autonomous Vehicles](https://mlsanity.com/paper/2212.06437/) (2022)
- [Safe Pontryagin Differentiable Programming](https://mlsanity.com/paper/2105.14937/) (2021)
- [Efficient differentiable quadratic programming layers: an ADMM approach](https://mlsanity.com/paper/2112.07464/) (2021)
- [Forward-backward envelope for the sum of two nonconvex functions: Further properties and nonmonotone line-search algorithms](https://mlsanity.com/paper/1606.06256/) (2016)
- [A Simple and Efficient Algorithm for Nonlinear Model Predictive Control](https://mlsanity.com/paper/1709.06487/) (2017)
- [Differentiable Nonlinear Model Predictive Control](https://mlsanity.com/paper/2505.01353/) (2025)
- [LeTO: Learning Constrained Visuomotor Policy with Differentiable Trajectory Optimization](https://mlsanity.com/paper/2401.17500/) (2024)
- [Alpaqa: A matrix-free solver for nonlinear MPC and large-scale nonconvex optimization](https://mlsanity.com/paper/2112.02370/) (2021)
- [Evaluation of MPC-based Imitation Learning for Human-like Autonomous Driving](https://mlsanity.com/paper/2211.12111/) (2022)
- [A Fully First-Order Layer for Differentiable Optimization](https://mlsanity.com/paper/2512.02494/) (2025)
- [Driving from Vision through Differentiable Optimal Control](https://mlsanity.com/paper/2403.15102/) (2024)
- [TurboMPC: Fast, Scalable, and Differentiable Model Predictive Control on the GPU](https://mlsanity.com/paper/2606.24039/) (2026)
- [PANDA: A Matrix-Free Differentiable NMPC Solver via Proximal Averaged Quasi-Newton with Adaptive Linesearch Algorithm](https://mlsanity.com/paper/2608.16280/) (2026)

## Abstract (arXiv)

Differentiable optimization brings the structural guarantees of mathematical optimization to network pipelines, allowing them to be trained end-to-end. However, its application remains challenging for nonconvex constrained problems, as existing differentiable solvers often suffer from limited modeling expressiveness due to their reliance on specialized problem structures, while also incurring substantial computation time and memory overhead in both the forward and backward passes. To address these challenges, we propose lapanda, a matrix-free differentiable solver for nonconvex optimization with general constraints. It reformulates the problem to a sequence of augmented Lagrangian subproblems, each handled by a first-order inner solver through a proximal averaged quasi-Newton algorithm with adaptive linesearch, thus enabling efficient forward optimization. We establish local well-posedness of the solution map and convergence of the outer iterations, and further derive a sensitivity alignment between the original problem and the final subproblem in the backward pass, demonstrating that the subproblem sensitivity, which can be computed efficiently in a matrix-free manner, provides a principled approximation to the exact optimizer sensitivity. We evaluate lapanda on nonconvex constrained Rosenbrock benchmarks, imitation learning with several representative constrained optimal control problems, and embedded robotic obstacle-avoidance tasks. Compared with state-of-the-art differentiable solvers, lapanda delivers substantial reductions in computation time and memory footprint while maintaining reliable constraint satisfaction and learning performance.