# JacNet: Learning Functions with Structured Jacobians

arXiv: https://arxiv.org/abs/2408.13237 · HTML: https://mlsanity.com/paper/2408.13237/

## Key facts

- Cited by 7 papers in the map (data through 14 Oct 2026).
- Builds on 27 of its 31 references that are in the map.
- First posted on arXiv 23 Aug 2024.

## Builds on (most cited first, 27 of 27)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Neural Ordinary Differential Equations](https://mlsanity.com/paper/1806.07366/) (2018)
- [Density estimation using Real NVP](https://mlsanity.com/paper/1605.08803/) (2016)
- [Glow: Generative Flow with Invertible 1x1 Convolutions](https://mlsanity.com/paper/1807.03039/) (2018)
- [NICE: Non-linear Independent Components Estimation](https://mlsanity.com/paper/1410.8516/) (2014)
- [Wasserstein GAN](https://mlsanity.com/paper/1701.07875/) (2017)
- [FFJORD: Free-form Continuous Dynamics for Scalable Reversible Generative Models](https://mlsanity.com/paper/1810.01367/) (2018)
- [Input Convex Neural Networks](https://mlsanity.com/paper/1609.07152/) (2016)
- [Invertible Residual Networks](https://mlsanity.com/paper/1811.00995/) (2018)
- [Optimizing Millions of Hyperparameters by Implicit Differentiation](https://mlsanity.com/paper/1911.02590/) (2019)
- [Learning with Opponent-Learning Awareness](https://mlsanity.com/paper/1709.04326/) (2017)
- [Sorting out Lipschitz function approximation](https://mlsanity.com/paper/1811.05381/) (2018)
- [Sobolev Training for Neural Networks](https://mlsanity.com/paper/1706.04859/) (2017)
- [i-RevNet: Deep Invertible Networks](https://mlsanity.com/paper/1802.07088/) (2018)
- [Self-Tuning Networks: Bilevel Optimization of Hyperparameters using Structured Best-Response Functions](https://mlsanity.com/paper/1903.03088/) (2019)
- [Stochastic Hyperparameter Optimization through Hypernetworks](https://mlsanity.com/paper/1802.09419/) (2018)
- [Using Large Language Models for Hyperparameter Optimization](https://mlsanity.com/paper/2312.04528/) (2023)
- [Training Data Attribution via Approximate Unrolled Differentiation](https://mlsanity.com/paper/2405.12186/) (2024)
- [On Implicit Bias in Overparameterized Bilevel Optimization](https://mlsanity.com/paper/2212.14032/) (2022)
- [Meta-Learning to Improve Pre-Training](https://mlsanity.com/paper/2111.01754/) (2021)
- [Input Convex Gradient Networks](https://mlsanity.com/paper/2111.12187/) (2021)
- [Understanding Neural Architecture Search Techniques](https://mlsanity.com/paper/1904.00438/) (2019)
- [Complex Momentum for Optimization in Games](https://mlsanity.com/paper/2102.08431/) (2021)
- [Lyapunov Exponents for Diversity in Differentiable Games](https://mlsanity.com/paper/2112.14570/) (2021)
- [Task Selection for AutoML System Evaluation](https://mlsanity.com/paper/2208.12754/) (2022)
- [Improving Hyperparameter Optimization with Checkpointed Model Weights](https://mlsanity.com/paper/2406.18630/) (2024)
- [Scalable Nested Optimization for Deep Learning](https://mlsanity.com/paper/2407.01526/) (2024)

## Built on by (most cited first, 7 of 7)

- [Addressable Memory for Video World Models](https://mlsanity.com/paper/2608.07408/) (2026)
- [Score Distillation Sampling for Audio: Source Separation, Synthesis, and Beyond](https://mlsanity.com/paper/2505.04621/) (2025)
- [Motion Attribution for Video Generation](https://mlsanity.com/paper/2601.08828/) (2026)
- [Characterizing control between interacting subsystems with deep Jacobian estimation](https://mlsanity.com/paper/2507.01946/) (2025)
- [Extended Neural Contractive Dynamical Systems: On Multiple Tasks and Riemannian Safety Regions](https://mlsanity.com/paper/2411.11405/) (2024)
- [Diffeomorphic Obstacle Avoidance for Contractive Dynamical Systems via Implicit Representations](https://mlsanity.com/paper/2504.18860/) (2025)
- [Universal Representation of Generalized Convex Functions and their Gradients](https://mlsanity.com/paper/2509.04477/) (2025)

## Abstract (arXiv)

Neural networks are trained to learn an approximate mapping from an input domain to a target domain. Incorporating prior knowledge about true mappings is critical to learning a useful approximation. With current architectures, it is challenging to enforce structure on the derivatives of the input-output mapping. We propose to use a neural network to directly learn the Jacobian of the input-output function, which allows easy control of the derivative. We focus on structuring the derivative to allow invertibility and also demonstrate that other useful priors, such as k-Lipschitz, can be enforced. Using this approach, we can learn approximations to simple functions that are guaranteed to be invertible and easily compute the inverse. We also show similar results for 1-Lipschitz functions.