Explicit Interaction Architectures for Dynamical Learning: A Controlled Study of Structural Inductive Bias
Not on the map. Why?
Abstract (arXiv)
We investigate a structure-first approach to dynamical learning in which the organization of stateful interactions is prescribed explicitly rather than left entirely to a generic recurrent parameterization. We introduce causal recurrent units built from an ordered sequence of local, state-modulated transformations. The construction is motivated by wave-based interaction models, but the units studied here do not impose scattering, passivity, or energy-balance constraints. Because fixed recurrent dynamics, designed reservoir topologies, readout-only learning, and recurrent depth are already well established, the empirical question is deliberately narrower: does the proposed interaction organization provide a useful inductive bias under controlled computational conditions? We compare a one-layer structured model, a two-layer structured model, and a generic echo-state network (ESN), all with 12 recurrent states and the same strictly linear ridge readout. Each model family receives the same random-search budget on calibration data that are disjoint from the final test data, after which the selected hyperparameters are frozen. On a custom nonlinear identification task, the one-layer structured model attains a mean validation NMSE of 2.76 x 10^{-4}, compared with 3.19 x 10^{-4} for the two-layer model and 3.94 x 10^{-4} for the ESN. On NARMA10 the ordering reverses: the ESN attains 0.312, compared with 0.348 and 0.357 for the one- and two-layer structured models. Thus, the proposed organization can be competitive and advantageous on one task, but it is not universally superior; moreover, recurrent depth does not provide a systematic benefit under matched state dimension. The results support a task-dependent interpretation of structural inductive bias and position the present architecture as a controlled precursor to stronger wave- and system-theoretic constructions.
Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 7 of its 21 references that are in the map. First posted on arXiv 17 Jun 2026.
Builds on
7 of 21 references.
References · 7
- 2019PyTorch: An Imperative Style, High-Performance Deep Learning Librarycited by 20,160
- 2018Neural Ordinary Differential Equationscited by 3,733
- 2018Relational inductive biases, deep learning, and graph networkscited by 1,341
- 2019Hamiltonian Neural Networkscited by 702
- 2021Exploiting Multiple Timescales in Hierarchical Echo State Networkscited by 11
- 2023Simple Cycle Reservoirs are Universalcited by 8
- 2019Reservoir Topology in Deep Echo State Networkscited by 6
Built on by
Not cited yet.