# TO-FLOW: Efficient Continuous Normalizing Flows with Temporal Optimization adjoint with Moving Speed

arXiv: https://arxiv.org/abs/2203.10335 · HTML: https://mlsanity.com/paper/2203.10335/

## Key facts

- Cited by 8 papers in the map (data through 14 Oct 2026).
- Builds on 57 of its 78 references that are in the map.
- First posted on arXiv 19 Mar 2022.

## Builds on (most cited first, 30 of 57)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Proximal Policy Optimization Algorithms](https://mlsanity.com/paper/1707.06347/) (2017)
- [Fashion-MNIST: a Novel Image Dataset for Benchmarking Machine Learning Algorithms](https://mlsanity.com/paper/1708.07747/) (2017)
- [Neural Ordinary Differential Equations](https://mlsanity.com/paper/1806.07366/) (2018)
- [Variational Inference with Normalizing Flows](https://mlsanity.com/paper/1505.05770/) (2015)
- [Density estimation using Real NVP](https://mlsanity.com/paper/1605.08803/) (2016)
- [Glow: Generative Flow with Invertible 1x1 Convolutions](https://mlsanity.com/paper/1807.03039/) (2018)
- [Normalizing Flows for Probabilistic Modeling and Inference](https://mlsanity.com/paper/1912.02762/) (2019)
- [NICE: Non-linear Independent Components Estimation](https://mlsanity.com/paper/1410.8516/) (2014)
- [Masked Autoregressive Flow for Density Estimation](https://mlsanity.com/paper/1705.07057/) (2017)
- [Improving Variational Inference with Inverse Autoregressive Flow](https://mlsanity.com/paper/1606.04934/) (2016)
- [Neural Spline Flows](https://mlsanity.com/paper/1906.04032/) (2019)
- [FFJORD: Free-form Continuous Dynamics for Scalable Reversible Generative Models](https://mlsanity.com/paper/1810.01367/) (2018)
- [Augmented Neural ODEs](https://mlsanity.com/paper/1904.01681/) (2019)
- [WaveGlow: A Flow-based Generative Network for Speech Synthesis](https://mlsanity.com/paper/1811.00002/) (2018)
- [Flow++: Improving Flow-Based Generative Models with Variational Dequantization and Architecture Design](https://mlsanity.com/paper/1902.00275/) (2019)
- [Equivariant Flows: Exact Likelihood Generative Learning for Symmetric Densities](https://mlsanity.com/paper/2006.02425/) (2020)
- [How to train your neural ODE: the world of Jacobian and kinetic regularization](https://mlsanity.com/paper/2002.02798/) (2020)
- [Neural Autoregressive Flows](https://mlsanity.com/paper/1804.00779/) (2018)
- [BSP-Net: Generating Compact Meshes via Binary Space Partitioning](https://mlsanity.com/paper/1911.06971/) (2019)
- [Neural Importance Sampling](https://mlsanity.com/paper/1808.03856/) (2018)
- [NASA: Neural Articulated Shape Approximation](https://mlsanity.com/paper/1912.03207/) (2019)
- [Riemannian Continuous Normalizing Flows](https://mlsanity.com/paper/2006.10605/) (2020)
- [OT-Flow: Fast and Accurate Continuous Normalizing Flows via Optimal Transport](https://mlsanity.com/paper/2006.00104/) (2020)
- [Normalizing Flows on Tori and Spheres](https://mlsanity.com/paper/2002.02428/) (2020)
- [GraphNVP: An Invertible Flow Model for Generating Molecular Graphs](https://mlsanity.com/paper/1905.11600/) (2019)
- [Noise Flow: Noise Modeling with Conditional Normalizing Flows](https://mlsanity.com/paper/1908.08453/) (2019)
- [Neural SDE: Stabilizing Neural ODE Networks with Stochastic Noise](https://mlsanity.com/paper/1906.02355/) (2019)
- [Unconstrained Monotonic Neural Networks](https://mlsanity.com/paper/1908.05164/) (2019)

## Built on by (most cited first, 8 of 8)

- [Flow Matching for Generative Modeling](https://mlsanity.com/paper/2210.02747/) (2022)
- [X-IL: Exploring the Design Space of Imitation Learning Policies](https://mlsanity.com/paper/2502.12330/) (2025)
- [Entropy-Informed Weighting Channel Normalizing Flow for Deep Generative Models](https://mlsanity.com/paper/2407.04958/) (2024)
- [A Minimum Variance Path Principle for Accurate and Stable Score-Based Density Ratio Estimation](https://mlsanity.com/paper/2602.00834/) (2026)
- [Sparse Variational Student-t Processes for Heavy-tailed Modeling](https://mlsanity.com/paper/2408.06699/) (2024)
- [Efficient Manifold-Constrained Neural ODE for High-Dimensional Datasets](https://mlsanity.com/paper/2510.04138/) (2025)
- [Flow matching on homogeneous spaces](https://mlsanity.com/paper/2603.24829/) (2026)
- [Instance Relation Learning Network with Label Knowledge Propagation for Few-shot Multi-label Intent Detection](https://mlsanity.com/paper/2510.07776/) (2025)

## Abstract (arXiv)

Continuous normalizing flows (CNFs) construct invertible mappings between an arbitrary complex distribution and an isotropic Gaussian distribution using Neural Ordinary Differential Equations (neural ODEs). It has not been tractable on large datasets due to the incremental complexity of the neural ODE training. Optimal Transport theory has been applied to regularize the dynamics of the ODE to speed up training in recent works. In this paper, a temporal optimization is proposed by optimizing the evolutionary time for forward propagation of the neural ODE training. In this appoach, we optimize the network weights of the CNF alternately with evolutionary time by coordinate descent. Further with temporal regularization, stability of the evolution is ensured. This approach can be used in conjunction with the original regularization approach. We have experimentally demonstrated that the proposed approach can significantly accelerate training without sacrifying performance over baseline models.