mlsanityData to 7 Oct 2026

Deep Declarative Networks: A New Hope

arXiv 1909.04866 ↗First posted 11 Sep 2019cs.LG cs.AI cs.CVcited by 77

Not on the map. Why?

Abstract (arXiv)

We explore a new class of end-to-end learnable models wherein data processing nodes (or network layers) are defined in terms of desired behavior rather than an explicit forward function. Specifically, the forward function is implicitly defined as the solution to a mathematical optimization problem. Consistent with nomenclature in the programming languages community, we name these models deep declarative networks. Importantly, we show that the class of deep declarative networks subsumes current deep learning models. Moreover, invoking the implicit function theorem, we show how gradients can be back-propagated through many declaratively defined data processing nodes thereby enabling end-to-end learning. We show how these declarative processing nodes can be implemented in the popular PyTorch deep learning software library allowing declarative and imperative nodes to co-exist within the same network. We also provide numerous insights and illustrative examples of declarative nodes and demonstrate their application for image and point cloud classification tasks.

Key facts. Cited by 77 papers in the map (data through 7 Oct 2026). First posted on arXiv 11 Sep 2019.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 14; 2022: 19; 2023: 16; 2024: 11; 2025: 10; 2026: 7.

77 papers.

Sort:

By area:

  1. 2022Transformers learn in-context by gradient descentcited by 714
  2. 2021Implicit Behavioral Cloningcited by 442
  3. 2023Uncovering mesa-optimization algorithms in Transformerscited by 94
  4. 2021A Review of the Gumbel-max Trick and its Extensions for Discrete Stochasticity in Machine Learningcited by 82
  5. 2023EPro-PnP: Generalized End-to-End Probabilistic Perspective-n-Points for Monocular Object Pose Estimationcited by 79
  6. 2021On Training Implicit Modelscited by 73
  7. 2023Learning How To Robustly Estimate Camera Pose in Endoscopic Videoscited by 49
  8. 2021Nonsmooth Implicit Differentiation for Machine Learning and Optimizationcited by 43
  9. 2025MesaNet: Sequence Modeling by Locally Optimal Test-Time Trainingcited by 37
  10. 2023Tracking and Mapping in Medical Computer Vision: A Reviewcited by 36
  11. 2021Enforcing Policy Feasibility Constraints through Differentiable Projection for Energy Optimizationcited by 31
  12. 2022DiffPoseNet: Direct Differentiable Camera Pose Estimationcited by 27
38 more
  1. 2023Scalable Optimal Transport Methods in Machine Learning: A Contemporary Surveycited by 27
  2. 2021Self-supervised Geometric Perceptioncited by 26
  3. 2022Beyond backpropagation: bilevel optimization through implicit differentiation and equilibrium propagationcited by 26
  4. 2022Alternating Differentiation for Optimization Layerscited by 22
  5. 2021To The Point: Correspondence-driven monocular 3D category reconstructioncited by 21
  6. 2023Deep Learning for Camera Calibration and Beyond: A Surveycited by 20
  7. 2025FSNet: Feasibility-Seeking Neural Network for Constrained Optimization with Guaranteescited by 19
  8. 2023Unsupervised Volumetric Animationcited by 18
  9. 2021A global convergence theory for deep ReLU implicit networks via over-parameterizationcited by 17
  10. 2022Robust Structured Declarative Classifiers for 3D Point Clouds: Defending Adversarial Attacks with Implicit Gradientscited by 17
  11. 2024LaneCPP: Continuous 3D Lane Detection using Physical Priorscited by 17
  12. 2022The least-control principle for local learning at equilibriumcited by 16
  13. 2024Imperative Learning: A Self-supervised Neuro-Symbolic Learning Framework for Robot Autonomycited by 16
  14. 2021DeepMesh: Differentiable Iso-Surface Extractioncited by 14
  15. 2022SC-wLS: Towards Interpretable Feed-forward Camera Re-localizationcited by 14
  16. 2023iSLAM: Imperative SLAMcited by 14
  17. 2024Guiding Neural Collapse: Optimising Towards the Nearest Simplex Equiangular Tight Framecited by 14
  18. 2022Certifiable 3D Object Pose Estimation: Foundations, Learning Models, and Self-Trainingcited by 12
  19. 2022Learning of Global Objective for Network Flow in Multi-Object Trackingcited by 11
  20. 2022ZippyPoint: Fast Interest Point Detection, Description, and Matching through Mixed Precision Discretizationcited by 10
  21. 2024Differentiation Through Black-Box Quadratic Programming Solverscited by 10
  22. 2024BPQP: A Differentiable Convex Optimization Framework for Efficient End-to-End Learningcited by 10
  23. 2023IKOL: Inverse kinematics optimization layer for 3D human pose and shape estimation via Gauss-Newton differentiationcited by 9
  24. 2021Generating Masks from Boxes by Mining Spatio-Temporal Consistencies in Videoscited by 8
  25. 2022Total Variation Optimization Layers for Computer Visioncited by 8
  26. 2023Revisiting Implicit Differentiation for Learning Problems in Optimal Controlcited by 8
  27. 2025End-to-End Probabilistic Framework for Learning with Hard Constraintscited by 8
  28. 2021Ensuring DNN Solution Feasibility for Optimization Problems with Convex Constraints and Its Application to DC Optimal Power Flow Problemscited by 6
  29. 2022Exploiting Problem Structure in Deep Declarative Networks: Two Case Studiescited by 6
  30. 2023A Correct-and-Certify Approach to Self-Supervise Object Pose Estimators via Ensemble Self-Trainingcited by 6
  31. 2021Bilevel Programs Meet Deep Learning: A Unifying View on Inference Learning Methodscited by 5
  32. 2022Improving Worst Case Visual Localization Coverage via Place-specific Sub-selection in Multi-camera Systemscited by 5
  33. 2022NeRFEditor: Differentiable Style Decomposition for Full 3D Scene Editingcited by 5
  34. 2023Deep Declarative Dynamic Time Warping for End-to-End Learning of Alignment Pathscited by 5
  35. 2024DiffTORI: Differentiable Trajectory Optimization for Deep Reinforcement and Imitation Learningcited by 5
  36. 2025Reversible Deep Equilibrium Modelscited by 5
  37. 2022Deep equilibrium models as estimators for continuous latent variablescited by 4
  38. 2025Global-Decision-Focused Neural ODEs for Proactive Grid Resilience Managementcited by 4

50 most cited of 77 All 77 (JSON)