mlsanityData to 14 Oct 2026

Survival is the Only Reward: Sustainable Self-Training Through Environment-Mediated Selection

arXiv 2601.12310 ↗First posted 18 Jan 2026cs.AIcited by 0

On the map: LLM alignment / RLHF / preference optimisation → Reward modeling and RLHF

Abstract (arXiv)

Self-training systems often degenerate due to the lack of an external criterion for judging data quality, leading to reward hacking and semantic drift. This paper provides a proof-of-concept system architecture for stable self-training under sparse external feedback and bounded memory, and empirically characterises its learning dynamics and failure modes. We introduce a self-training architecture in which learning is mediated exclusively by environmental viability, rather than by reward, objective functions, or externally defined fitness criteria. Candidate behaviours are executed under real resource constraints, and only those whose environmental effects both persist and preserve the possibility of future interaction are propagated. The environment does not provide semantic feedback, dense rewards, or task-specific supervision; selection operates solely through differential survival of behaviours as world-altering events, making proxy optimisation impossible and rendering reward-hacking evolutionarily unstable. Analysis of semantic dynamics shows that improvement arises primarily through the persistence of effective and repeatable strategies under a regime of consolidation and pruning, a paradigm we refer to as negative-space learning (NSL), and that models develop meta-learning strategies (such as deliberate experimental failure in order to elicit informative error messages) without explicit instruction. This work establishes that environment-grounded selection enables sustainable open-ended self-improvement, offering a viable path toward more robust and generalisable autonomous systems without reliance on human-curated data or complex reward shaping.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in Reward modeling and RLHF, one of 10 directions in LLM alignment / RLHF / preference optimisation. First posted on arXiv 18 Jan 2026.

Builds on

0 of 0 references.

No references here yet.

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2026Learning with Rare Success but Rich Feedback via Reflection-Enhanced Self-Distillation
  2. 2026Teaching LLMs to Self-Evolve: Cultivating Core Meta-Skills with Reinforcement Learning
  3. 2026Self-Play Only Evolves When Self-Synthetic Pipeline Ensures Learnable Information Gain
  4. 2026Survive or Collapse: The Asymmetric Roles of Data Gating and Reward Grounding in Self-Play RL
  5. 2026Intrinsic Motivation in Reinforcement Learning: A Research Agenda for Adaptive Self-Organisation
  6. 2026EvoBelief: Falsifiable World Models for Self-Improving LLM Agents
  7. 2026Recursive Experiential-Working Memory Evolution for Long-Horizon Agent Harnesses
  8. 2025Context and Diversity Matter: The Emergence of In-Context Learning in World Models