# CoLoRA: Continuous low-rank adaptation for reduced implicit neural modeling of parameterized partial differential equations

arXiv: https://arxiv.org/abs/2402.14646 · HTML: https://mlsanity.com/paper/2402.14646/

## Key facts

- Cited by 14 papers in the map (data through 14 Oct 2026).
- Builds on 34 of its 105 references that are in the map.
- First posted on arXiv 22 Feb 2024.

## Builds on (most cited first, 30 of 34)

- [LoRA: Low-Rank Adaptation of Large Language Models](https://mlsanity.com/paper/2106.09685/) (2021)
- [Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networks](https://mlsanity.com/paper/1703.03400/) (2017)
- [Fourier Neural Operator for Parametric Partial Differential Equations](https://mlsanity.com/paper/2010.08895/) (2020)
- [Implicit Neural Representations with Periodic Activation Functions](https://mlsanity.com/paper/2006.09661/) (2020)
- [DeepONet: Learning nonlinear operators for identifying differential equations based on the universal approximation theorem of operators](https://mlsanity.com/paper/1910.03193/) (2019)
- [Intrinsic Dimensionality Explains the Effectiveness of Language Model Fine-Tuning](https://mlsanity.com/paper/2012.13255/) (2020)
- [Message Passing Neural PDE Solvers](https://mlsanity.com/paper/2202.03376/) (2022)
- [Measuring the Intrinsic Dimension of Objective Landscapes](https://mlsanity.com/paper/1804.08838/) (2018)
- [Factorized Fourier Neural Operators](https://mlsanity.com/paper/2111.13802/) (2021)
- [Lift & Learn: Physics-informed machine learning for large-scale nonlinear dynamical systems](https://mlsanity.com/paper/1912.08177/) (2019)
- [Operator Learning with Neural Fields: Tackling PDEs on General Geometries](https://mlsanity.com/paper/2306.07266/) (2023)
- [A fast and accurate physics-informed neural network reduced order model with shallow masked autoencoder](https://mlsanity.com/paper/2009.11990/) (2020)
- [A Mathematical Guide to Operator Learning](https://mlsanity.com/paper/2312.14688/) (2023)
- [Continuous PDE Dynamics Forecasting with Implicit Neural Representations](https://mlsanity.com/paper/2209.14855/) (2022)
- [Initialization and Regularization of Factorized Neural Layers](https://mlsanity.com/paper/2105.01029/) (2021)
- [CROM: Continuous Reduced-Order Modeling of PDEs Using Implicit Neural Representations](https://mlsanity.com/paper/2206.02607/) (2022)
- [Neural Galerkin Schemes with Active Learning for High-Dimensional Evolution Equations](https://mlsanity.com/paper/2203.01360/) (2022)
- [Evolutional Deep Neural Network](https://mlsanity.com/paper/2103.09959/) (2021)
- [Parameterized Neural Ordinary Differential Equations: Applications to Computational Physics Problems](https://mlsanity.com/paper/2010.14685/) (2020)
- [Hypernetwork-based Meta-Learning for Low-Rank Physics-Informed Neural Networks](https://mlsanity.com/paper/2310.09528/) (2023)
- [Low-rank lottery tickets: finding efficient low-rank neural networks via matrix differential equations](https://mlsanity.com/paper/2205.13571/) (2022)
- [HyperPINN: Learning parameterized differential equations with physics-informed hypernetworks](https://mlsanity.com/paper/2111.01008/) (2021)
- [Generalizable Implicit Neural Representations via Instance Pattern Composers](https://mlsanity.com/paper/2211.13223/) (2022)
- [Neural Implicit Flow: a mesh-agnostic dimensionality reduction paradigm of spatio-temporal data](https://mlsanity.com/paper/2204.03216/) (2022)
- [Implicit Neural Spatial Representations for Time-dependent PDEs](https://mlsanity.com/paper/2210.00124/) (2022)
- [Quadratic Approximation Manifold for Mitigating the Kolmogorov Barrier in Nonlinear Projection-Based Model Order Reduction](https://mlsanity.com/paper/2204.02462/) (2022)
- [Randomized Sparse Neural Galerkin Schemes for Solving Evolution Equations with Deep Networks](https://mlsanity.com/paper/2310.04867/) (2023)
- [The Neural Network shifted-Proper Orthogonal Decomposition: a Machine Learning Approach for Non-linear Reduction of Hyperbolic Equations](https://mlsanity.com/paper/2108.06558/) (2021)
- [Evolve Smoothly, Fit Consistently: Learning Smooth Latent Dynamics For Advection-Dominated Systems](https://mlsanity.com/paper/2301.10391/) (2023)
- [Progressive construction of a parametric reduced-order model for PDE-constrained optimization](https://mlsanity.com/paper/1407.7618/) (2014)

## Built on by (most cited first, 14 of 14)

- [LeMON: Learning to Learn Multi-Operator Networks](https://mlsanity.com/paper/2408.16168/) (2024)
- [Parametric model reduction of mean-field and stochastic systems via higher-order action matching](https://mlsanity.com/paper/2410.12000/) (2024)
- [Learning of Population Dynamics: Inverse Optimization Meets JKO Scheme](https://mlsanity.com/paper/2506.01502/) (2025)
- [Sequential-in-time training of nonlinear parametrizations for solving time-dependent partial differential equations](https://mlsanity.com/paper/2404.01145/) (2024)
- [Learning Lagrangian Interaction Dynamics with Sampling-Based Model Order Reduction](https://mlsanity.com/paper/2407.03925/) (2024)
- [System stabilization with policy optimization on unstable latent manifolds](https://mlsanity.com/paper/2407.06418/) (2024)
- [Adapting Projection-Based Reduced-Order Models using Projected Gaussian Process](https://mlsanity.com/paper/2410.14090/) (2024)
- [Low-Rank Evolutionary Deep Neural Networks via Adaptive Tangent-Space Reduction](https://mlsanity.com/paper/2509.16395/) (2025)
- [Filtered Neural Galerkin model reduction schemes for efficient propagation of initial condition uncertainties in digital twins](https://mlsanity.com/paper/2511.00670/) (2025)
- [An adaptive data sampling strategy for stabilizing dynamical systems via controller inference](https://mlsanity.com/paper/2506.01816/) (2025)
- [Beyond Static Models: Hypernetworks for Adaptive and Generalizable Forecasting in Complex Parametric Dynamical Systems](https://mlsanity.com/paper/2506.19609/) (2025)
- [ODELoRA: Training Low-Rank Adaptation by Solving Ordinary Differential Equations](https://mlsanity.com/paper/2602.07479/) (2026)
- [SMORE: Stability-Promoting Mesh-Agnostic Model Reduction for Time-Dependent PDEs](https://mlsanity.com/paper/2609.33205/) (2026)
- [PDE-JEPA: Predictive Representation Learning of Latent Dynamics Modeling for Parametric PDEs](https://mlsanity.com/paper/2609.34715/) (2026)

## Abstract (arXiv)

This work introduces reduced models based on Continuous Low Rank Adaptation (CoLoRA) that pre-train neural networks for a given partial differential equation and then continuously adapt low-rank weights in time to rapidly predict the evolution of solution fields at new physics parameters and new initial conditions. The adaptation can be either purely data-driven or via an equation-driven variational approach that provides Galerkin-optimal approximations. Because CoLoRA approximates solution fields locally in time, the rank of the weights can be kept small, which means that only few training trajectories are required offline so that CoLoRA is well suited for data-scarce regimes. Predictions with CoLoRA are orders of magnitude faster than with classical methods and their accuracy and parameter efficiency is higher compared to other neural network approaches.