# Towards autonomous artificial agents with an active self: modeling sense of control in situated action

arXiv: https://arxiv.org/abs/2112.05577 · HTML: https://mlsanity.com/paper/2112.05577/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 7 of its 60 references that are in the map.
- First posted on arXiv 10 Dec 2021.

## Builds on (most cited first, 7 of 7)

- [The Option-Critic Architecture](https://mlsanity.com/paper/1609.05140/) (2016)
- [Learning and Transfer of Modulated Locomotor Controllers](https://mlsanity.com/paper/1610.05182/) (2016)
- [On Reinforcement Learning for Full-length Game of StarCraft](https://mlsanity.com/paper/1809.09095/) (2018)
- [Learning Representations in Model-Free Hierarchical Reinforcement Learning](https://mlsanity.com/paper/1810.10096/) (2018)
- [From semantics to execution: Integrating action planning with reinforcement learning for robotic causal problem-solving](https://mlsanity.com/paper/1905.09683/) (2019)
- [A predictive processing model of perception and action for self-other distinction](https://mlsanity.com/paper/1810.09879/) (2018)
- [From Crystallized Adaptivity to Fluid Adaptivity in Deep Reinforcement Learning -- Insights from Biological Systems on Adaptive Flexibility](https://mlsanity.com/paper/1908.05348/) (2019)

## Abstract (arXiv)

In this paper we present a computational modeling account of an active self in artificial agents. In particular we focus on how an agent can be equipped with a sense of control and how it arises in autonomous situated action and, in turn, influences action control. We argue that this requires laying out an embodied cognitive model that combines bottom-up processes (sensorimotor learning and fine-grained adaptation of control) with top-down processes (cognitive processes for strategy selection and decision-making). We present such a conceptual computational architecture based on principles of predictive processing and free energy minimization. Using this general model, we describe how a sense of control can form across the levels of a control hierarchy and how this can support action control in an unpredictable environment. We present an implementation of this model as well as first evaluations in a simulated task scenario, in which an autonomous agent has to cope with un-/predictable situations and experiences corresponding sense of control. We explore different model parameter settings that lead to different ways of combining low-level and high-level action control. The results show the importance of appropriately weighting information in situations where the need for low/high-level action control varies and they demonstrate how the sense of control can facilitate this.