# Representing Knowledge as Predictions (and State as Knowledge)

arXiv: https://arxiv.org/abs/2112.06336 · HTML: https://mlsanity.com/paper/2112.06336/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 1 of its 38 references that are in the map.
- First posted on arXiv 12 Dec 2021.

## Builds on (most cited first, 1 of 1)

- [Temporal-Difference Networks](https://mlsanity.com/paper/1504.05539/) (2015)

## Built on by (most cited first, 1 of 1)

- [Towards a Robust Soft Baby Robot With Rich Interaction Ability for Advanced Machine Learning Algorithms](https://mlsanity.com/paper/2404.08093/) (2024)

## Abstract (arXiv)

This paper shows how a single mechanism allows knowledge to be constructed layer by layer directly from an agent's raw sensorimotor stream. This mechanism, the General Value Function (GVF) or "forecast," captures high-level, abstract knowledge as a set of predictions about existing features and knowledge, based exclusively on the agent's low-level senses and actions. Thus, forecasts provide a representation for organizing raw sensorimotor data into useful abstractions over an unlimited number of layers--a long-sought goal of AI and cognitive science. The heart of this paper is a detailed thought experiment providing a concrete, step-by-step formal illustration of how an artificial agent can build true, useful, abstract knowledge from its raw sensorimotor experience alone. The knowledge is represented as a set of layered predictions (forecasts) about the agent's observed consequences of its actions. This illustration shows twelve separate layers: the lowest consisting of raw pixels, touch and force sensors, and a small number of actions; the higher layers increasing in abstraction, eventually resulting in rich knowledge about the agent's world, corresponding roughly to doorways, walls, rooms, and floor plans. I then argue that this general mechanism may allow the representation of a broad spectrum of everyday human knowledge.