# Making sense of sensory input

arXiv: https://arxiv.org/abs/1910.02227 · HTML: https://mlsanity.com/paper/1910.02227/

## Key facts

- Cited by 29 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 5 Oct 2019.

## Built on by (most cited first, 29 of 29)

- [Abstraction and Analogy-Making in Artificial Intelligence](https://mlsanity.com/paper/2102.10717/) (2021)
- [Neural Production Systems: Learning Rule-Governed Visual Dynamics](https://mlsanity.com/paper/2103.01937/) (2021)
- [Inductive logic programming at 30](https://mlsanity.com/paper/2102.10556/) (2021)
- [Abstraction for Deep Reinforcement Learning](https://mlsanity.com/paper/2202.05839/) (2022)
- [Learning MDL logic programs from noisy data](https://mlsanity.com/paper/2308.09393/) (2023)
- [Predicate Invention by Learning From Failures](https://mlsanity.com/paper/2104.14426/) (2021)
- [Learning Symbolic Representations Through Joint GEnerative and DIscriminative Training](https://mlsanity.com/paper/2304.11357/) (2023)
- [Learning logic programs by combining programs](https://mlsanity.com/paper/2206.01614/) (2022)
- [Relational decomposition for program synthesis](https://mlsanity.com/paper/2408.12212/) (2024)
- [Learning First-Order Rules with Differentiable Logic Program Semantics](https://mlsanity.com/paper/2204.13570/) (2022)
- [Learning logic programs by discovering where not to search](https://mlsanity.com/paper/2202.09806/) (2022)
- [Learning big logical rules by joining small rules](https://mlsanity.com/paper/2401.16215/) (2024)
- [Variable Assignment Invariant Neural Networks for Learning Logic Programs](https://mlsanity.com/paper/2408.10709/) (2024)
- [The Artificial Scientist: Logicist, Emergentist, and Universalist Approaches to Artificial General Intelligence](https://mlsanity.com/paper/2110.01831/) (2021)
- [Intensional Artificial Intelligence: From Symbol Emergence to Explainable and Empathetic AI](https://mlsanity.com/paper/2104.11573/) (2021)
- [Adapting the Function Approximation Architecture in Online Reinforcement Learning](https://mlsanity.com/paper/2106.09776/) (2021)
- [Learning programs with magic values](https://mlsanity.com/paper/2208.03238/) (2022)
- [Relational program synthesis with numerical reasoning](https://mlsanity.com/paper/2210.00764/) (2022)
- [Visual Graph Question Answering with ASP and LLMs for Language Parsing](https://mlsanity.com/paper/2502.09211/) (2025)
- [Learning logic programs through divide, constrain, and conquer](https://mlsanity.com/paper/2109.07818/) (2021)
- [Learning logic programs by discovering higher-order abstractions](https://mlsanity.com/paper/2308.08334/) (2023)
- [Learning logic programs by explaining their failures](https://mlsanity.com/paper/2102.12551/) (2021)
- [Parallel Constraint-Driven Inductive Logic Programming](https://mlsanity.com/paper/2109.07132/) (2021)
- [Language-Based Causal Representation Learning](https://mlsanity.com/paper/2207.05259/) (2022)
- [A Recursive Bateson-Inspired Model for the Generation of Semantic Formal Concepts from Spatial Sensory Data](https://mlsanity.com/paper/2307.08087/) (2023)
- [An Empirical Comparison of Cost Functions in Inductive Logic Programming](https://mlsanity.com/paper/2503.07554/) (2025)
- [Honey, I shrunk the hypothesis space (through logical preprocessing)](https://mlsanity.com/paper/2506.06739/) (2025)
- [Symmetry breaking for inductive logic programming](https://mlsanity.com/paper/2508.06263/) (2025)
- [Continual learning and refinement of causal models through dynamic predicate invention](https://mlsanity.com/paper/2602.17217/) (2026)

## Abstract (arXiv)

This paper attempts to answer a central question in unsupervised learning: what does it mean to "make sense" of a sensory sequence? In our formalization, making sense involves constructing a symbolic causal theory that both explains the sensory sequence and also satisfies a set of unity conditions. The unity conditions insist that the constituents of the causal theory -- objects, properties, and laws -- must be integrated into a coherent whole. On our account, making sense of sensory input is a type of program synthesis, but it is unsupervised program synthesis. Our second contribution is a computer implementation, the Apperception Engine, that was designed to satisfy the above requirements. Our system is able to produce interpretable human-readable causal theories from very small amounts of data, because of the strong inductive bias provided by the unity conditions. A causal theory produced by our system is able to predict future sensor readings, as well as retrodict earlier readings, and impute (fill in the blanks of) missing sensory readings, in any combination. We tested the engine in a diverse variety of domains, including cellular automata, rhythms and simple nursery tunes, multi-modal binding problems, occlusion tasks, and sequence induction intelligence tests. In each domain, we test our engine's ability to predict future sensor values, retrodict earlier sensor values, and impute missing sensory data. The engine performs well in all these domains, significantly out-performing neural net baselines. We note in particular that in the sequence induction intelligence tests, our system achieved human-level performance. This is notable because our system is not a bespoke system designed specifically to solve intelligence tests, but a general-purpose system that was designed to make sense of any sensory sequence.