mlsanityData to 14 Oct 2026

F-Adapter: Frequency-Adaptive Parameter-Efficient Fine-Tuning in Scientific Machine Learning

arXiv 2509.23173 ↗First posted 27 Sep 2025cs.LGcited by 3

Not on the map. Why?

Abstract (arXiv)

Parameter-efficient fine-tuning (PEFT) of powerful pre-trained models for complex downstream tasks has proven effective in vision and language processing, yet this paradigm remains unexplored in scientific machine learning, where the objective is to model complex physical systems. We conduct the first systematic study of PEFT for pre-trained Large Operator Models (LOMs) obtained by scaling variants of Fourier Neural Operator. First, we observe that the widely used Low-Rank Adaptation (LoRA) yields markedly poorer performance on LOMs than Adapter tuning. Then, we further theoretically establish that stacked LoRA incurs a depth-amplified lower bound on approximation error within Fourier layers, whereas adapters retain universal approximation capacity and, by concentrating parameters on energy-dominant low-frequency modes, attain exponentially decaying error with bottleneck width in the Fourier domain. Motivated by the robust empirical gains of adapters and by our theoretical characterization of PDE solutions as spectrally sparse, we introduce Frequency-Adaptive Adapter (F-Adapter). F-Adapter allocates adapter capacity based on spectral complexity, assigning higher-dimension modules to low-frequency components and lower-dimension modules to high-frequency components. Our F-Adapters establish state-of-the-art (SOTA) results on multiple challenging 3D Navier-Stokes benchmarks, markedly enhancing both generalization and spectral fidelity over LoRA and other PEFT techniques commonly used in LLMs. To the best of our knowledge, this work is the first to explore PEFT for scientific machine-learning and establishes F-Adapter as an effective paradigm for this domain.

Key facts. Cited by 3 papers in the map (data through 14 Oct 2026). Builds on 48 of its 62 references that are in the map. First posted on arXiv 27 Sep 2025.

Builds on

48 of 62 references.

References · 48

  1. 2019Parameter-Efficient Transfer Learning for NLPcited by 4,236
  2. 2021Prefix-Tuning: Optimizing Continuous Prompts for Generationcited by 3,915
  3. 2021The Power of Scale for Parameter-Efficient Prompt Tuningcited by 3,795
  4. 2020Fourier Neural Operator for Parametric Partial Differential Equationscited by 2,908
  5. 2021BitFit: Simple Parameter-efficient Fine-tuning for Transformer-based Masked Language-modelscited by 1,245
  6. 2022Few-Shot Parameter-Efficient Fine-Tuning is Better and Cheaper than In-Context Learningcited by 896
  7. 2022FourCastNet: A Global Data-driven High-resolution Weather Model using Adaptive Fourier Neural Operatorscited by 776
  8. 2022AdaptFormer: Adapting Vision Transformers for Scalable Visual Recognitioncited by 719
  9. 2024KAN: Kolmogorov-Arnold Networkscited by 649
  10. 2022Fourier Neural Operator with Learned Deformations for PDEs on General Geometriescited by 427
  11. 2022PDEBENCH: An Extensive Benchmark for Scientific Machine Learningcited by 413
  12. 2023AdaLoRA: Adaptive Budget Allocation for Parameter-Efficient Fine-Tuningcited by 357
36 more
  1. 2020Parameter-Efficient Transfer Learning with Diff Pruningcited by 352
  2. 2021Factorized Fourier Neural Operatorscited by 249
  3. 2021Adaptive Fourier Neural Operators: Efficient Token Mixers for Transformerscited by 248
  4. 2023Spherical Fourier Neural Operators: Learning Stable Dynamics on the Spherecited by 233
  5. 2023Neural Operators for Accelerating Scientific Simulations and Designcited by 221
  6. 2024Poseidon: Efficient Foundation Models for PDEscited by 200
  7. 2024DPOT: Auto-Regressive Denoising Operator Transformer for Large-Scale PDE Pre-Trainingcited by 151
  8. 2023Towards Foundation Models for Scientific Machine Learning: Characterizing Scaling and Transfer Behaviorcited by 131
  9. 2023Scalable Transformer for PDE Surrogate Modelingcited by 128
  10. 2024HydraLoRA: An Asymmetric LoRA Architecture for Efficient Fine-Tuningcited by 123
  11. 2024Chebyshev Polynomial-Based Kolmogorov-Arnold Networks: An Efficient Architecture for Nonlinear Function Approximationcited by 118
  12. 2023Multiple Physics Pretraining for Physical Surrogate Modelscited by 98
  13. 2022Wavelet neural operator: a neural operator for parametric partial differential equationscited by 78
  14. 2023PINNsFormer: A Transformer-Based Framework For Physics-Informed Neural Networkscited by 64
  15. 2024Pretraining Codomain Attention Neural Operators for Solving Multiphysics PDEscited by 59
  16. 2024SVFT: Parameter-Efficient Fine-Tuning with Singular Vectorscited by 58
  17. 2022Polyhistor: Parameter-Efficient Multi-Task Adaptation for Dense Vision Taskscited by 52
  18. 2022Derivative-Informed Neural Operator: An Efficient Framework for High-Dimensional Parametric Derivative Learningcited by 44
  19. 2023Hypernetwork-based Meta-Learning for Low-Rank Physics-Informed Neural Networkscited by 44
  20. 2023Domain Agnostic Fourier Neural Operatorscited by 38
  21. 2025RandLoRA: Full-rank parameter-efficient fine-tuning of large modelscited by 35
  22. 2024UPS: Efficiently Building Foundation Models for PDE Solving via Cross-Modal Adaptationcited by 27
  23. 2022FiT: Parameter Efficient Few-shot Transfer Learning for Personalized and Federated Image Classificationcited by 26
  24. 2025Densely Connected Parameter-Efficient Tuning for Referring Image Segmentationcited by 26
  25. 2022Incremental Spatial and Spectral Learning of Neural Operators for Solving Large-Scale PDEscited by 25
  26. 2023Cross-Lingual Transfer with Target Language-Ready Task Adapterscited by 15
  27. 2025Transfer Learning in Physics-Informed Neural Networks: Full Fine-Tuning, Lightweight Fine-Tuning, and Low-Rank Adaptationcited by 15
  28. 2024Strategies for Pretraining Neural Operatorscited by 14
  29. 2024RST-LoRA: A Discourse-Aware Low-Rank Adaptation for Long Document Abstractive Summarizationcited by 11
  30. 2025Anant-Net: Breaking the Curse of Dimensionality with Scalable and Interpretable Neural Surrogate for High-Dimensional PDEscited by 11
  31. 2024OmniArch: Building Foundation Model For Scientific Computingcited by 9
  32. 2023Mini but Mighty: Finetuning ViTs with Mini Adapterscited by 6
  33. 2025TensorGRaD: Tensor Gradient Robust Decomposition for Memory-Efficient Neural Operator Trainingcited by 6
  34. 2024Pretraining a Neural Operator in Lower Dimensionscited by 5
  35. 2024Global-local Fourier Neural Operator for Accelerating Coronal Magnetic Field Modelcited by 4
  36. 2025AC-PKAN: Attention-Enhanced and Chebyshev Polynomial-Based Physics-Informed Kolmogorov-Arnold Networkscited by 2

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 3.

3 papers.

  1. 2026Iterative Refinement Neural Operators are Learned Fixed-Point Solvers: A Principled Approach to Spectral Bias Mitigationcited by 2
  2. 2026Adapting Automotive Aerodynamics Surrogates to New Vehicle Families via Transfer Learningcited by 1
  3. 2026A Model-Agnostic Physics-Guided Adapter for Few-Shot Transfer of Coastal Flood Prediction Models to Unseen Regionscited by 0