# Explicit Interaction Architectures for Dynamical Learning: A Controlled Study of Structural Inductive Bias

arXiv: https://arxiv.org/abs/2606.19101 · HTML: https://mlsanity.com/paper/2606.19101/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 7 of its 21 references that are in the map.
- First posted on arXiv 17 Jun 2026.

## Builds on (most cited first, 7 of 7)

- [PyTorch: An Imperative Style, High-Performance Deep Learning Library](https://mlsanity.com/paper/1912.01703/) (2019)
- [Neural Ordinary Differential Equations](https://mlsanity.com/paper/1806.07366/) (2018)
- [Relational inductive biases, deep learning, and graph networks](https://mlsanity.com/paper/1806.01261/) (2018)
- [Hamiltonian Neural Networks](https://mlsanity.com/paper/1906.01563/) (2019)
- [Exploiting Multiple Timescales in Hierarchical Echo State Networks](https://mlsanity.com/paper/2101.04223/) (2021)
- [Simple Cycle Reservoirs are Universal](https://mlsanity.com/paper/2308.10793/) (2023)
- [Reservoir Topology in Deep Echo State Networks](https://mlsanity.com/paper/1909.11022/) (2019)

## Abstract (arXiv)

We investigate a structure-first approach to dynamical learning in which the organization of stateful interactions is prescribed explicitly rather than left entirely to a generic recurrent parameterization. We introduce causal recurrent units built from an ordered sequence of local, state-modulated transformations. The construction is motivated by wave-based interaction models, but the units studied here do not impose scattering, passivity, or energy-balance constraints. Because fixed recurrent dynamics, designed reservoir topologies, readout-only learning, and recurrent depth are already well established, the empirical question is deliberately narrower: does the proposed interaction organization provide a useful inductive bias under controlled computational conditions? We compare a one-layer structured model, a two-layer structured model, and a generic echo-state network (ESN), all with 12 recurrent states and the same strictly linear ridge readout. Each model family receives the same random-search budget on calibration data that are disjoint from the final test data, after which the selected hyperparameters are frozen. On a custom nonlinear identification task, the one-layer structured model attains a mean validation NMSE of 2.76 x 10^{-4}, compared with 3.19 x 10^{-4} for the two-layer model and 3.94 x 10^{-4} for the ESN. On NARMA10 the ordering reverses: the ESN attains 0.312, compared with 0.348 and 0.357 for the one- and two-layer structured models. Thus, the proposed organization can be competitive and advantageous on one task, but it is not universally superior; moreover, recurrent depth does not provide a systematic benefit under matched state dimension. The results support a task-dependent interpretation of structural inductive bias and position the present architecture as a controlled precursor to stronger wave- and system-theoretic constructions.