mlsanityData to 7 Oct 2026

Discovery by Dreaming: Cross-Domain Recombination in Artificial Memory

arXiv 2607.16256 ↗First posted 28 Jun 2026cs.LG cs.AI cs.IRcited by 0

Not on the map. Why?

Abstract (arXiv)

Dreams splice together people, places, and times that never met. Neuroscience suggests this recombination is not noise, but a function driving insight and creative discovery. This reframes memory consolidation: rather than merely defending against forgetting, its measurable value lies in recombining knowledge across experiences that have not yet co-occurred. We test this directly by isolating the recombinatory-replay mechanism and implementing it in two architecturally unrelated systems: a LoRA fine-tuning pipeline (DREAMS) and a symbolic engine replaying structured knowledge objects (SAPIENCE). Both systems converge on the same finding: cross-domain consolidation creates value, while within-domain rehearsal does not. The symbolic arm surfaces novel cross-domain connections at 85.7%, a +21 percentage point (pp) gain over baseline. The neural arm improves overall by +5.64 pp, but on subtasks explicitly requiring cross-domain transfer (like unseen math reasoning on GSM8K), gains reach +14.5 pp. This effect is a genuine property of the weights--not a prompt artifact--as prepending the same material in-context to a 671B-parameter model actually reverses the gain. We validate this prediction against documented discoveries across 50,000 real papers and state a falsifiable hippocampal-recording prediction to distinguish recombination from rehearsal. Ultimately, this principle is substrate-general, tracking real discovery at scale. Reading the literature teaches a model to recall what it has seen, but producing discovery requires a separate offline phase that recombines knowledge across domains--the computational analog of dreaming. Consolidation is not for remembering, but for discovering.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 21 of its 56 references that are in the map. First posted on arXiv 28 Jun 2026.

Builds on

21 of 56 references.

References · 21

  1. 2022Training language models to follow instructions with human feedbackcited by 15,712
  2. 2021LoRA: Low-Rank Adaptation of Large Language Modelscited by 14,084
  3. 2015Distilling the Knowledge in a Neural Networkcited by 10,076
  4. 2020Retrieval-Augmented Generation for Knowledge-Intensive NLP Taskscited by 9,160
  5. 2016Overcoming catastrophic forgetting in neural networkscited by 4,718
  6. 2022Self-Instruct: Aligning Language Models with Self-Generated Instructionscited by 2,674
  7. 2023Self-RAG: Learning to Retrieve, Generate, and Critique through Self-Reflectioncited by 1,800
  8. 2016Progressive Neural Networkscited by 1,332
  9. 2017Continual Learning with Deep Generative Replaycited by 1,152
  10. 2018Efficient Lifelong Learning with A-GEMcited by 1,044
  11. 2017PackNet: Adding Multiple Tasks to a Single Network by Iterative Pruningcited by 882
  12. 2023FinGPT: Open-Source Financial Large Language Modelscited by 244
9 more
  1. 2018The Geometry of Culture: Analyzing Meaning through Word Embeddingscited by 87
  2. 2024BioMedLM: A 2.7B Parameter Language Model Trained On Biomedical Textcited by 85
  3. 2025Self-Adapting Language Modelscited by 71
  4. 2023Accelerating science with human-aware artificial intelligencecited by 27
  5. 2013Tradition and Innovation in Scientists' Research Strategiescited by 26
  6. 2019The Diversity-Innovation Paradox in Sciencecited by 21
  7. 2021New Directions in Science Emerge from Disconnection and Discordcited by 8
  8. 2026Unlocking LLM Creativity in Science through Analogical Reasoningcited by 6
  9. 2026Language Models Need Sleep: Learning to Self-Modify and Consolidate Memoriescited by 1

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2026The World Model Remembers, the Actor Forgets: Dream Rehearsal for Continual Model-Based RL
  2. 2025GENESIS: A Generative Model of Episodic-Semantic Interaction
  3. 2026Dream2Learn: Structured Generative Dreaming for Continual Learning
  4. 2022Towards biologically plausible Dreaming and Planning in recurrent spiking networks
  5. 2026Useful Memories Become Faulty When Continuously Updated by LLMs
  6. 2025AI Meets Brain: Memory Systems from Cognitive Neuroscience to Autonomous Agents
  7. 2026Memory Beyond Recall: A Dual-Process Cognitive Memory System for Self-Evolving LLM Agents
  8. 2026Auto-Dreamer: Learning Offline Memory Consolidation for Language Agents