Habits and goals in synergy: a variational Bayesian framework for behavior
Not on the map. Why?
Abstract (arXiv)
How to behave efficiently and flexibly is a central problem for understanding biological agents and creating intelligent embodied AI. It has been well known that behavior can be classified as two types: reward-maximizing habitual behavior, which is fast while inflexible; and goal-directed behavior, which is flexible while slow. Conventionally, habitual and goal-directed behaviors are considered handled by two distinct systems in the brain. Here, we propose to bridge the gap between the two behaviors, drawing on the principles of variational Bayesian theory. We incorporate both behaviors in one framework by introducing a Bayesian latent variable called "intention". The habitual behavior is generated by using prior distribution of intention, which is goal-less; and the goal-directed behavior is generated by the posterior distribution of intention, which is conditioned on the goal. Building on this idea, we present a novel Bayesian framework for modeling behaviors. Our proposed framework enables skill sharing between the two kinds of behaviors, and by leveraging the idea of predictive coding, it enables an agent to seamlessly generalize from habitual to goal-directed behavior without requiring additional training. The proposed framework suggests a fresh perspective for cognitive science and embodied AI, highlighting the potential for greater integration between habitual and goal-directed behaviors.
Key facts. Cited by 3 papers in the map (data through 7 Oct 2026). Builds on 51 of its 137 references that are in the map. First posted on arXiv 11 Apr 2023.
Builds on
51 of 137 references.
References · 51
- 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
- 2020Language Models are Few-Shot Learnerscited by 32,199
- 2021Swin Transformer: Hierarchical Vision Transformer using Shifted Windowscited by 12,082
- 2018Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actorcited by 4,635
- 2014Learning Phrase Representations using RNN Encoder-Decoder for Statistical Machine Translationcited by 4,016
- 2021On the Opportunities and Risks of Foundation Modelscited by 3,961
- 2019Dream to Control: Learning Behaviors by Latent Imaginationcited by 1,598
- 2019Mastering Atari, Go, Chess and Shogi by Planning with a Learned Modelcited by 1,327
- 2018Learning Latent Dynamics for Planning from Pixelscited by 1,221
- 2018Soft Actor-Critic Algorithms and Applicationscited by 1,179
- 2019Meta-World: A Benchmark and Evaluation for Multi-Task and Meta Reinforcement Learningcited by 1,168
- 2017Hindsight Experience Replaycited by 1,096
38 more
- 2016Deep Variational Information Bottleneckcited by 915
- 2015Deep Learning and the Information Bottleneck Principlecited by 793
- 2018Recurrent World Models Facilitate Policy Evolutioncited by 761
- 2018Diversity is All You Need: Learning Skills without a Reward Functioncited by 734
- 2019Model-Based Reinforcement Learning for Ataricited by 495
- 2018Reinforcement Learning and Control as Probabilistic Inference: Tutorial and Reviewcited by 466
- 2015A Recurrent Latent Variable Model for Sequential Datacited by 362
- 2018Quantifying Generalization in Reinforcement Learningcited by 345
- 2016Ladder Variational Autoencoderscited by 315
- 2019Dynamics-Aware Unsupervised Discovery of Skillscited by 308
- 2016Variational Intrinsic Controlcited by 262
- 2020Very Deep VAEs Generalize Autoregressive Models and Can Outperform Them on Imagescited by 261
- 2017Automatic Goal Generation for Reinforcement Learning Agentscited by 252
- 2019Stochastic Latent Actor-Critic: Deep Reinforcement Learning with a Latent Variable Modelcited by 228
- 2017Asymmetric Actor Critic for Image-Based Robot Learningcited by 202
- 2019A Survey of Reinforcement Learning Informed by Natural Languagecited by 154
- 2022Goal-Conditioned Reinforcement Learning: Problems and Solutionscited by 145
- 2021Discovering and Achieving Goals via World Modelscited by 131
- 2018Deep Variational Reinforcement Learning for POMDPscited by 122
- 2018Variational Option Discovery Algorithmscited by 121
- 2021Recurrent Model-Free RL Can Be a Strong Baseline for Many POMDPscited by 115
- 2016Towards an integration of deep learning and neurosciencecited by 77
- 2020Deep active inference agents using Monte-Carlo methodscited by 72
- 2019Deep Active Inference as Variational Policy Gradientscited by 52
- 2019Variational Recurrent Models for Solving Partially Observable Control Taskscited by 46
- 2017Deep Active Inferencecited by 44
- 2020On the Relationship Between Active Inference and Control as Inferencecited by 42
- 2020Deep Reinforcement Learning and its Neuroscientific Implicationscited by 33
- 2020Action and Perception as Divergence Minimizationcited by 31
- 2020Continual Learning of Control Primitives: Skill Discovery via Reset-Gamescited by 29
- 2017Combining Model-Based and Model-Free Updates for Trajectory-Centric Reinforcement Learningcited by 27
- 2021Contrastive Active Inferencecited by 21
- 2022Intelligent problem-solving as integrated hierarchical reinforcement learningcited by 21
- 2019Learning Dynamics Model in Reinforcement Learning by Incorporating the Long Term Futurecited by 17
- 2018One Big Net For Everythingcited by 13
- 2019Intrinsically Motivated Discovery of Diverse Patterns in Self-Organizing Systemscited by 13
- 2020Goal-Directed Planning for Habituated Agents by Active Inference Using a Variational Recurrent Neural Networkcited by 5
- 2022Goal-directed Planning and Goal Understanding by Active Inference: Evaluation Through Simulated and Physical Robot Experimentscited by 4
50 most cited of 51 All 51 (JSON)
Built on by
Citing papers per year: 2023: 1; 2024: 0; 2025: 2.
3 papers.