# Learning, Planning, and Control in a Monolithic Neural Event Inference Architecture

arXiv: https://arxiv.org/abs/1809.07412 · HTML: https://mlsanity.com/paper/1809.07412/

## Key facts

- Cited by 8 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 19 Sep 2018.

## Built on by (most cited first, 8 of 8)

- [Sparsely Changing Latent States for Prediction and Planning in Partially Observable Domains](https://mlsanity.com/paper/2110.15949/) (2021)
- [Branching Time Active Inference: empirical study and complexity class analysis](https://mlsanity.com/paper/2111.11276/) (2021)
- [Goal-directed Planning and Goal Understanding by Active Inference: Evaluation Through Simulated and Physical Robot Experiments](https://mlsanity.com/paper/2202.09976/) (2022)
- [Gradient-based inference of abstract task representations for generalization in neural networks](https://mlsanity.com/paper/2407.17356/) (2024)
- [Inference of Affordances and Active Motor Control in Simulated Agents](https://mlsanity.com/paper/2202.11532/) (2022)
- [Flexible task abstractions emerge in linear networks with fast and bounded units](https://mlsanity.com/paper/2411.03840/) (2024)
- [Binding Dancers Into Attractors](https://mlsanity.com/paper/2206.02558/) (2022)
- [A Review of Mechanistic Models of Event Comprehension](https://mlsanity.com/paper/2409.18992/) (2024)

## Abstract (arXiv)

We introduce REPRISE, a REtrospective and PRospective Inference SchEme, which learns temporal event-predictive models of dynamical systems. REPRISE infers the unobservable contextual event state and accompanying temporal predictive models that best explain the recently encountered sensorimotor experiences retrospectively. Meanwhile, it optimizes upcoming motor activities prospectively in a goal-directed manner. Here, REPRISE is implemented by a recurrent neural network (RNN), which learns temporal forward models of the sensorimotor contingencies generated by different simulated dynamic vehicles. The RNN is augmented with contextual neurons, which enable the encoding of distinct, but related, sensorimotor dynamics as compact event codes. We show that REPRISE concurrently learns to separate and approximate the encountered sensorimotor dynamics: it analyzes sensorimotor error signals adapting both internal contextual neural activities and connection weight values. Moreover, we show that REPRISE can exploit the learned model to induce goal-directed, model-predictive control, that is, approximate active inference: Given a goal state, the system imagines a motor command sequence optimizing it with the prospective objective to minimize the distance to the goal. The RNN activities thus continuously imagine the upcoming future and reflect on the recent past, optimizing the predictive model, the hidden neural state activities, and the upcoming motor activities. As a result, event-predictive neural encodings develop, which allow the invocation of highly effective and adaptive goal-directed sensorimotor control.