Factorization Regret mediates compositional generalization in latent space

Are there still barriers to generalization once all relevant variables are known? We consider the challenge of generalizing to a novel combination of task-relevant latent variables in the Cognitive Gridworld, a stationary Partially Observable Markov Decision Process where observations are generated by latent variables with parametric interactions. We establish Factorization Regret as an information-theoretic quantity measuring the contribution of latent variable interactions to task performance. Using this metric, we explain the sub-optimality of Echo State Networks and uncover a failure mode whereby confidence becomes decoupled from accuracy. This suggests that utilizing interactions between relevant variables is a non-trivial capability. We next introduce Representation Classification Chains to show that latent variable interactions and how to infer the variables' values can be learned together through Reinforcement Learning. Solving this variational inference problem enables generalization to novel combinations of relevant variables. Lastly, we demonstrate that once interactions are learned, imagined experience is sufficient for zero-shot control. In summary, we develop and motivate a minimal, theoretically grounded setting for research into generalization from experience.