The Equilibrium Is the Initialization: Lazy Identity Collapse in Physics-Structured Deep Equilibrium Reasoning
Not on the map. Why?
Abstract (arXiv)
Deep equilibrium models promise input-adaptive implicit computation: harder problems should demand more solver iterations, and the solved equilibrium should encode the result of genuine iterative inference. We report a cautionary study of a port-Hamiltonian DEQ with a learned initialization on two reasoning tasks -- ProofWriter entailment over frozen DeBERTa embeddings and a BFS-verified graph-reachability benchmark -- in which the implicit computation is a silent no-op. Across tasks, seeds, and controlled ablation arms, the solved equilibrium equals the solver's start point to numerical precision, and bypassing the solver entirely changes test accuracy by +0.00 percentage points in 18 of 19 training runs. Controlled interventions falsify the tempting explanation: removing the anchoring term reproduces every result, and retraining with noise-decoupled starts yields a solver that converges to the noisy start while the decoder learns to ignore it. The single escaping run diverges instead (|h*-z0|=171), producing a co-adapted noise channel whose removal improves accuracy. Iteration counts are uncorrelated with ground-truth difficulty (r=0.009), and the full apparatus never outperforms a two-layer MLP on either task. We trace the mechanism to gradient starvation along two distinct routes, show that the standard zeroing ablation is confounded and gives wildly seed-dependent answers where the correct substitution test gives a stable zero, and distill a four-test diagnostic protocol for auditing claimed implicit computation. All experiments run on a single free Colab GPU; code, raw logs, and analysis scripts are released.
Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 23 of its 28 references that are in the map. First posted on arXiv 13 Jul 2026.
Builds on
23 of 28 references.
References · 23
- 2018Neural Ordinary Differential Equationscited by 3,779
- 2020Shortcut Learning in Deep Neural Networkscited by 1,969
- 2016Understanding intermediate layers using linear classifier probescited by 1,353
- 2019Hamiltonian Neural Networkscited by 712
- 2019Designing and Interpreting Probes with Control Taskscited by 640
- 2019Deep Equilibrium Modelscited by 615
- 2016Adaptive Computation Time for Recurrent Neural Networkscited by 506
- 2025Scaling up Test-Time Compute with Latent Reasoning: A Recurrent Depth Approachcited by 415
- 2020Lagrangian Neural Networkscited by 398
- 2020ProofWriter: Generating Implications, Proofs, and Abductive Statements over Natural Languagecited by 324
- 2020Gradient Starvation: A Learning Proclivity in Neural Networkscited by 267
- 2025Hierarchical Reasoning Modelcited by 177
11 more
- 2021PonderNet: Learning to Pondercited by 168
- 2020Multiscale Deep Equilibrium Modelscited by 151
- 2023Looped Transformers are Better at Learning Learning Algorithmscited by 149
- 2019Implicit Deep Learningcited by 124
- 2021Can You Learn an Algorithm? Generalizing from Easy to Hard Problems with Recurrent Networkscited by 117
- 2022The CLRS Algorithmic Reasoning Benchmarkcited by 106
- 2020Monotone operator equilibrium networkscited by 100
- 2021JFB: Jacobian-Free Backpropagation for Implicit Networkscited by 86
- 2021Stabilizing Equilibrium Models by Jacobian Regularizationcited by 72
- 2021Port-Hamiltonian Neural Networks for Learning Explicit Time-Dependent Dynamical Systemscited by 67
- 2022Path Independent Equilibrium Models Can Better Exploit Test-Time Computationcited by 38
Built on by
Not cited yet.