mlsanityData to 14 Oct 2026

Objective Value Change and Shape-Based Accelerated Optimization for the Neural Network Approximation

arXiv 2508.20290 ↗First posted 27 Aug 2025cs.LG cs.AI cs.NAcited by 0

Not on the map. Why?

Abstract (arXiv)

This paper introduce a novel metric of an objective function f, we say VC (value change) to measure the difficulty and approximation affection when conducting an neural network approximation task, and it numerically supports characterizing the local performance and behavior of neural network approximation. Neural networks often suffer from unpredictable local performance, which can hinder their reliability in critical applications. VC addresses this issue by providing a quantifiable measure of local value changes in network behavior, offering insights into the stability and performance for achieving the neural-network approximation. We investigate some fundamental theoretical properties of VC and identified two intriguing phenomena in neural network approximation: the VC-tendency and the minority-tendency. These trends respectively characterize how pointwise errors evolve in relation to the distribution of VC during the approximation process.In addition, we propose a novel metric based on VC, which measures the distance between two functions from the perspective of variation. Building upon this metric, we further propose a new preprocessing framework for neural network approximation. Numerical results including the real-world experiment and the PDE-related scientific problem support our discovery and pre-processing acceleration method.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 19 of its 42 references that are in the map. First posted on arXiv 27 Aug 2025.

Builds on

19 of 42 references.

References · 19

  1. 2021Swin Transformer: Hierarchical Vision Transformer using Shifted Windowscited by 12,139
  2. 2018Neural Ordinary Differential Equationscited by 3,779
  3. 2018Neural Tangent Kernel: Convergence and Generalization in Neural Networkscited by 2,621
  4. 2020When and why PINNs fail to train: A neural tangent kernel perspectivecited by 634
  5. 2016Error bounds for approximations with deep ReLU networkscited by 574
  6. 2021Physics-informed neural networks (PINNs) for fluid mechanics: A reviewcited by 473
  7. 2019Frequency Principle: Fourier Analysis Sheds Light on Deep Neural Networkscited by 368
  8. 2017The Expressive Power of Neural Networks: A View from the Widthcited by 320
  9. 2016Benefits of depth in neural networkscited by 250
  10. 2020On the linearity of large non-linear models: when and why the tangent kernel is constantcited by 129
  11. 2019Machine learning for molecular simulationcited by 110
  12. 2024FedBiOT: LLM Local Fine-tuning in Federated Learning without Full Modelcited by 74
7 more
  1. 2022Overview frequency principle/spectral bias in deep learningcited by 69
  2. 2025Simulating Three-dimensional Turbulence with Physics-informed Neural Networkscited by 14
  3. 2022Towards Efficient and Stable K-Asynchronous Federated Learning with Unbounded Stale Gradients on Non-IID Datacited by 9
  4. 2024Is the neural tangent kernel of PINNs deep learning general partial differential equations always convergent ?cited by 5
  5. 2024Review of Mathematical Optimization in Federated Learningcited by 3
  6. 2023Derivative-Free Optimization with Transformed Objective Functions (DFOTO) and the Algorithm Based on the Least Frobenius Norm Updating Quadratic Modelcited by 1
  7. 2023Sufficient Conditions for Error Distance Reduction in the (ℓ2)-norm Trust Region between Minimizers of Local Nonconvex Multivariate Quadratic Approximatescited by 1

Built on by

Not cited yet.