# CosmoCore Affective Dream-Replay Reinforcement Learning for Code Generation

arXiv: https://arxiv.org/abs/2510.18895 · HTML: https://mlsanity.com/paper/2510.18895/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 13 of its 24 references that are in the map.
- First posted on arXiv 20 Oct 2025.

## Builds on (most cited first, 13 of 13)

- [Prioritized Experience Replay](https://mlsanity.com/paper/1511.05952/) (2015)
- [CodeRL: Mastering Code Generation through Pretrained Models and Deep Reinforcement Learning](https://mlsanity.com/paper/2207.01780/) (2022)
- [RLEF: Grounding Code LLMs in Execution Feedback with Reinforcement Learning](https://mlsanity.com/paper/2410.02089/) (2024)
- [CodeElo: Benchmarking Competition-level Code Generation of LLMs with Human-comparable Elo Ratings](https://mlsanity.com/paper/2501.01257/) (2025)
- [ReCode: Robustness Evaluation of Code Generation Models](https://mlsanity.com/paper/2212.10264/) (2022)
- [StepCoder: Improve Code Generation with Reinforcement Learning from Compiler Feedback](https://mlsanity.com/paper/2402.01391/) (2024)
- [LLM Collaboration With Multi-Agent Reinforcement Learning](https://mlsanity.com/paper/2508.04652/) (2025)
- [Actor Prioritized Experience Replay](https://mlsanity.com/paper/2209.00532/) (2022)
- [CodeBoost: Boosting Code LLMs by Squeezing Knowledge from Code Snippets with RL](https://mlsanity.com/paper/2508.05242/) (2025)
- [A Review of Neuroscience-Inspired Machine Learning](https://mlsanity.com/paper/2403.18929/) (2024)
- [Emotions in the Loop: A Survey of Affective Computing for Emotional Support](https://mlsanity.com/paper/2505.01542/) (2025)
- [BIMRL: Brain Inspired Meta Reinforcement Learning](https://mlsanity.com/paper/2210.16530/) (2022)
- [Edge Intelligence with Spiking Neural Networks](https://mlsanity.com/paper/2507.14069/) (2025)

## Built on by (most cited first, 1 of 1)

- [CosmoCore-Evo: Evolutionary Dream-Replay Reinforcement Learning for Adaptive Code Generation](https://mlsanity.com/paper/2512.21351/) (2025)

## Abstract (arXiv)

We introduce CosmoCore, a neuroscience-inspired reinforcement learning (RL) architecture that integrates affective signals to enhance code generation in large language models (LLMs). Motivated by human and animal learning where embarrassment from mistakes drives rapid correction, as observed in training a puppy to avoid repeating errors after a single scolding CosmoCore tags code generation trajectories with valence and surprise using a lightweight multi-layer perceptron (MLP). High-negative valence (cringe) episodes, such as buggy code outputs, are prioritized in a Dream Queue for five-fold replay during off-policy updates, while low-surprise successes are pruned to prevent overconfidence and buffer bloat. Evaluated on code generation benchmarks like HumanEval and BigCodeBench, alongside simulations with a custom data pipeline environment, CosmoCore reduces hallucinated code (e.g., syntax errors or logical bugs) by 48% and accelerates self-correction by 45%. Local experiments using Hugging Face models in a PySpark environment validate these gains, with code snippets provided for replication. Ablations confirm valence tagging boosts curiosity in exploration, and pruning mitigates inefficiency. This framework extends RL from human feedback (RLHF) for more emotionally aware code assistants, with applications in IDEs and data pipelines. Code and the custom mini-world simulation are released.