mlsanityData to 7 Oct 2026

Circuit-Inspired High-Order Neural Networks with Unified Neural Dynamics Modeling for PDE Solving and Visual Perception

arXiv 2603.23977 ↗First posted 25 Mar 2026cs.LG cs.AIcited by 0

Not on the map. Why?

Abstract (arXiv)

Deep networks often rely on architectural heuristics to shape representation evolution, limiting their ability to model data governed by intrinsic dynamics. We present the Circuit-inspired High-Order Neural Network (CHONN), a modular framework that treats representation evolution as a latent potential process and increases its effective order through Kirchhoff-inspired cascade composition. A single Kirchhoff Neural Cell implements a stable first-order update, while serially composed cells form higher-order dynamical operators within one block. This construction is interpretable, numerically stable and compatible with common neural backbones. Theoretical analysis shows that cascaded cells induce end-to-end high-order operators, and controlled experiments demonstrate that intra-block high-order construction differs from generic depth stacking, especially on derivative-sensitive measures. Across steady-state operator learning, long-horizon physical forecasting and ImageNet-1K recognition, CHONN improves structural fidelity, rollout stability and visual representation learning. These results identify high-order circuit composition as a general principle for neural dynamics modeling.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 23 of its 41 references that are in the map. First posted on arXiv 25 Mar 2026.

Builds on

23 of 41 references.

References · 23

  1. 2017Attention Is All You Needcited by 59,932
  2. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  3. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 29,915
  4. 2015U-Net: Convolutional Networks for Biomedical Image Segmentationcited by 19,604
  5. 2021Swin Transformer: Hierarchical Vision Transformer using Shifted Windowscited by 12,082
  6. 2023Mamba: Linear-Time Sequence Modeling with Selective State Spacescited by 4,616
  7. 2018Neural Ordinary Differential Equationscited by 3,733
  8. 2022A ConvNet for the 2020scited by 3,545
  9. 2020Fourier Neural Operator for Parametric Partial Differential Equationscited by 2,806
  10. 2021Efficiently Modeling Long Sequences with Structured State Spacescited by 2,424
  11. 2020GLU Variants Improve Transformercited by 1,724
  12. 2024VMamba: Visual State Space Modelcited by 955
11 more
  1. 2022Simplified State Space Layers for Sequence Modelingcited by 776
  2. 2022PDEBENCH: An Extensive Benchmark for Scientific Machine Learningcited by 396
  3. 2024MambaVision: A Hybrid Mamba-Transformer Vision Backbonecited by 222
  4. 2023Convolutional Neural Operators for robust and accurate learning of PDEscited by 207
  5. 2024EfficientVMamba: Atrous Selective Scan for Light Weight Visual Mambacited by 146
  6. 2024Weak baselines and reporting biases lead to overoptimism in machine learning for fluid-related partial differential equationscited by 87
  7. 2022Deep transfer operator learning for partial differential equations under conditional shiftcited by 83
  8. 2021Closed-form Continuous-time Neural Modelscited by 65
  9. 2023Length Extrapolation of Transformers: A Survey from the Perspective of Positional Encodingcited by 32
  10. 2022Neural Integral Equationscited by 23
  11. 2024Positional Encoding Helps Recurrent Neural Networks Handle a Large Vocabularycited by 5

Built on by

Not cited yet.