mlsanityData to 7 Oct 2026

CoLoRA: Continuous low-rank adaptation for reduced implicit neural modeling of parameterized partial differential equations

arXiv 2402.14646 ↗First posted 22 Feb 2024cs.LG cs.NA math.NAcited by 12

Not on the map. Why?

Abstract (arXiv)

This work introduces reduced models based on Continuous Low Rank Adaptation (CoLoRA) that pre-train neural networks for a given partial differential equation and then continuously adapt low-rank weights in time to rapidly predict the evolution of solution fields at new physics parameters and new initial conditions. The adaptation can be either purely data-driven or via an equation-driven variational approach that provides Galerkin-optimal approximations. Because CoLoRA approximates solution fields locally in time, the rank of the weights can be kept small, which means that only few training trajectories are required offline so that CoLoRA is well suited for data-scarce regimes. Predictions with CoLoRA are orders of magnitude faster than with classical methods and their accuracy and parameter efficiency is higher compared to other neural network approaches.

Key facts. Cited by 12 papers in the map (data through 7 Oct 2026). Builds on 34 of its 105 references that are in the map. First posted on arXiv 22 Feb 2024.

Builds on

34 of 105 references.

References · 34

  1. 2021LoRA: Low-Rank Adaptation of Large Language Modelscited by 14,084
  2. 2017Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networkscited by 4,861
  3. 2020Fourier Neural Operator for Parametric Partial Differential Equationscited by 2,806
  4. 2020Implicit Neural Representations with Periodic Activation Functionscited by 2,214
  5. 2019DeepONet: Learning nonlinear operators for identifying differential equations based on the universal approximation theorem of operatorscited by 1,965
  6. 2020Intrinsic Dimensionality Explains the Effectiveness of Language Model Fine-Tuningcited by 658
  7. 2022Message Passing Neural PDE Solverscited by 368
  8. 2018Measuring the Intrinsic Dimension of Objective Landscapescited by 364
  9. 2021Factorized Fourier Neural Operatorscited by 237
  10. 2019Lift & Learn: Physics-informed machine learning for large-scale nonlinear dynamical systemscited by 96
  11. 2020A fast and accurate physics-informed neural network reduced order model with shallow masked autoencodercited by 80
  12. 2023Operator Learning with Neural Fields: Tackling PDEs on General Geometriescited by 80
22 more
  1. 2023A Mathematical Guide to Operator Learningcited by 74
  2. 2022Continuous PDE Dynamics Forecasting with Implicit Neural Representationscited by 71
  3. 2021Initialization and Regularization of Factorized Neural Layerscited by 58
  4. 2022CROM: Continuous Reduced-Order Modeling of PDEs Using Implicit Neural Representationscited by 56
  5. 2022Neural Galerkin Schemes with Active Learning for High-Dimensional Evolution Equationscited by 55
  6. 2021Evolutional Deep Neural Networkcited by 48
  7. 2020Parameterized Neural Ordinary Differential Equations: Applications to Computational Physics Problemscited by 45
  8. 2022Low-rank lottery tickets: finding efficient low-rank neural networks via matrix differential equationscited by 42
  9. 2023Hypernetwork-based Meta-Learning for Low-Rank Physics-Informed Neural Networkscited by 42
  10. 2021HyperPINN: Learning parameterized differential equations with physics-informed hypernetworkscited by 38
  11. 2022Generalizable Implicit Neural Representations via Instance Pattern Composerscited by 38
  12. 2022Neural Implicit Flow: a mesh-agnostic dimensionality reduction paradigm of spatio-temporal datacited by 37
  13. 2022Implicit Neural Spatial Representations for Time-dependent PDEscited by 35
  14. 2022Quadratic Approximation Manifold for Mitigating the Kolmogorov Barrier in Nonlinear Projection-Based Model Order Reductioncited by 29
  15. 2023Randomized Sparse Neural Galerkin Schemes for Solving Evolution Equations with Deep Networkscited by 18
  16. 2021The Neural Network shifted-Proper Orthogonal Decomposition: a Machine Learning Approach for Non-linear Reduction of Hyperbolic Equationscited by 14
  17. 2023Evolve Smoothly, Fit Consistently: Learning Smooth Latent Dynamics For Advection-Dominated Systemscited by 14
  18. 2014Progressive construction of a parametric reduced-order model for PDE-constrained optimizationcited by 13
  19. 2023Coupling parameter and particle dynamics for adaptive sampling in Neural Galerkin schemescited by 10
  20. 2021On-the-fly Reduced Order Modeling of Passive and Reactive Species via Time-Dependent Manifoldscited by 6
  21. 2023Lookahead data-gathering strategies for online adaptive model reduction of transport-dominated problemscited by 5
  22. 2023Nonlinear embeddings for conserving Hamiltonians and other quantities with Neural Galerkin schemescited by 3

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 6; 2025: 5; 2026: 1.

12 papers.

  1. 2024LeMON: Learning to Learn Multi-Operator Networkscited by 13
  2. 2024Parametric model reduction of mean-field and stochastic systems via higher-order action matchingcited by 8
  3. 2024Sequential-in-time training of nonlinear parametrizations for solving time-dependent partial differential equationscited by 7
  4. 2025Learning of Population Dynamics: Inverse Optimization Meets JKO Schemecited by 6
  5. 2024Learning Lagrangian Interaction Dynamics with Sampling-Based Model Order Reductioncited by 2
  6. 2024System stabilization with policy optimization on unstable latent manifoldscited by 2
  7. 2024Adapting Projection-Based Reduced-Order Models using Projected Gaussian Processcited by 2
  8. 2025Low-Rank Evolutionary Deep Neural Networks via Adaptive Tangent-Space Reductioncited by 1
  9. 2025Filtered Neural Galerkin model reduction schemes for efficient propagation of initial condition uncertainties in digital twinscited by 1
  10. 2025An adaptive data sampling strategy for stabilizing dynamical systems via controller inferencecited by 0
  11. 2025Beyond Static Models: Hypernetworks for Adaptive and Generalizable Forecasting in Complex Parametric Dynamical Systemscited by 0
  12. 2026ODELoRA: Training Low-Rank Adaptation by Solving Ordinary Differential Equationscited by 0