Discovery by Dreaming: Cross-Domain Recombination in Artificial Memory
Not on the map. Why?
Abstract (arXiv)
Dreams splice together people, places, and times that never met. Neuroscience suggests this recombination is not noise, but a function driving insight and creative discovery. This reframes memory consolidation: rather than merely defending against forgetting, its measurable value lies in recombining knowledge across experiences that have not yet co-occurred. We test this directly by isolating the recombinatory-replay mechanism and implementing it in two architecturally unrelated systems: a LoRA fine-tuning pipeline (DREAMS) and a symbolic engine replaying structured knowledge objects (SAPIENCE). Both systems converge on the same finding: cross-domain consolidation creates value, while within-domain rehearsal does not. The symbolic arm surfaces novel cross-domain connections at 85.7%, a +21 percentage point (pp) gain over baseline. The neural arm improves overall by +5.64 pp, but on subtasks explicitly requiring cross-domain transfer (like unseen math reasoning on GSM8K), gains reach +14.5 pp. This effect is a genuine property of the weights--not a prompt artifact--as prepending the same material in-context to a 671B-parameter model actually reverses the gain. We validate this prediction against documented discoveries across 50,000 real papers and state a falsifiable hippocampal-recording prediction to distinguish recombination from rehearsal. Ultimately, this principle is substrate-general, tracking real discovery at scale. Reading the literature teaches a model to recall what it has seen, but producing discovery requires a separate offline phase that recombines knowledge across domains--the computational analog of dreaming. Consolidation is not for remembering, but for discovering.
Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 21 of its 56 references that are in the map. First posted on arXiv 28 Jun 2026.
Builds on
21 of 56 references.
References · 21
- 2022Training language models to follow instructions with human feedbackcited by 15,712
- 2021LoRA: Low-Rank Adaptation of Large Language Modelscited by 14,084
- 2015Distilling the Knowledge in a Neural Networkcited by 10,076
- 2020Retrieval-Augmented Generation for Knowledge-Intensive NLP Taskscited by 9,160
- 2016Overcoming catastrophic forgetting in neural networkscited by 4,718
- 2022Self-Instruct: Aligning Language Models with Self-Generated Instructionscited by 2,674
- 2023Self-RAG: Learning to Retrieve, Generate, and Critique through Self-Reflectioncited by 1,800
- 2016Progressive Neural Networkscited by 1,332
- 2017Continual Learning with Deep Generative Replaycited by 1,152
- 2018Efficient Lifelong Learning with A-GEMcited by 1,044
- 2017PackNet: Adding Multiple Tasks to a Single Network by Iterative Pruningcited by 882
- 2023FinGPT: Open-Source Financial Large Language Modelscited by 244
9 more
- 2018The Geometry of Culture: Analyzing Meaning through Word Embeddingscited by 87
- 2024BioMedLM: A 2.7B Parameter Language Model Trained On Biomedical Textcited by 85
- 2025Self-Adapting Language Modelscited by 71
- 2023Accelerating science with human-aware artificial intelligencecited by 27
- 2013Tradition and Innovation in Scientists' Research Strategiescited by 26
- 2019The Diversity-Innovation Paradox in Sciencecited by 21
- 2021New Directions in Science Emerge from Disconnection and Discordcited by 8
- 2026Unlocking LLM Creativity in Science through Analogical Reasoningcited by 6
- 2026Language Models Need Sleep: Learning to Self-Modify and Consolidate Memoriescited by 1
Built on by
Not cited yet.
Similar, not cited
Close in wording, no citation either way
- 2026The World Model Remembers, the Actor Forgets: Dream Rehearsal for Continual Model-Based RL
- 2025GENESIS: A Generative Model of Episodic-Semantic Interaction
- 2026Dream2Learn: Structured Generative Dreaming for Continual Learning
- 2022Towards biologically plausible Dreaming and Planning in recurrent spiking networks
- 2026Useful Memories Become Faulty When Continuously Updated by LLMs
- 2025AI Meets Brain: Memory Systems from Cognitive Neuroscience to Autonomous Agents
- 2026Memory Beyond Recall: A Dual-Process Cognitive Memory System for Self-Evolving LLM Agents
- 2026Auto-Dreamer: Learning Offline Memory Consolidation for Language Agents