mlsanityData to 14 Oct 2026

Online Reward-Punishment Learning from Fixed-Channel Perceptual Event Streams without Environment Rewards

arXiv 2606.18963 ↗First posted 17 Jun 2026cs.LGcited by 0

Not on the map. Why?

Abstract (arXiv)

We study online reward-punishment learning when the environment provides no scalar reward or evaluative label. At each step the agent receives only a fixed-channel perceptual packet, and quantities such as pain, energy, contact, damage, or cognitive error are treated as perceptual dimensions whose valence must be inferred from transition consequences. OHIRL separates four roles: M_psi learns next-packet prediction, D_omega models residual dynamics, C_eta is a fixed internal post-transition trajectory evaluator, and B_xi learns to use the resulting value evidence for later policy updates and action scoring. C_eta uses a recovery-positive and persistence/growth-negative residual-regulation orientation; a coefficient-origin audit shows that equal-unit, raw-equal, and random monotone variants preserve more than 92% of the released top-action rankings, while sign inversion preserves 0%. The reward-free protocol exposes observation transitions while withholding environment rewards, delayed external evaluators, success labels, and action-goodness labels. A conditional error decomposition separates B_xi evidence-estimation error from residual policy-optimization error. In a 2x2-XOR packet task, medicine and chili acquire opposite value under visual XOR contexts, and the same pain or spice increase can be positive or negative depending on consequence structure; B_xi reaches 0.952 balanced reward-sign accuracy. In a full online-interleaved audit, M_psi reaches holdout R2=0.907, B_xi reaches 0.940 sign accuracy, and the policy reaches 0.979 optimal-action accuracy, while immediate packet scores, prediction-error rewards, shuffled targets, zero reward, and error-reduction controls collapse. Hidden-reward CartPole and Taxi controls, public-context no-leakage audits, and module-role ablations further test information boundaries and component necessity.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 13 of its 34 references that are in the map. First posted on arXiv 17 Jun 2026.

Builds on

13 of 34 references.

References · 13

  1. 2017Proximal Policy Optimization Algorithmscited by 15,215
  2. 2017Deep reinforcement learning from human preferencescited by 3,912
  3. 2023Mastering Diverse Domains through World Modelscited by 1,253
  4. 2017Curiosity-driven Exploration by Self-supervised Predictioncited by 1,194
  5. 2018Diversity is All You Need: Learning Skills without a Reward Functioncited by 752
  6. 2016Unifying Count-Based Exploration and Intrinsic Motivationcited by 639
  7. 2020Planning to Explore via Self-Supervised World Modelscited by 368
  8. 2016#Exploration: A Study of Count-Based Exploration for Deep Reinforcement Learningcited by 309
  9. 2020Reward-Free Exploration for Reinforcement Learningcited by 177
  10. 2024Exploration-Driven Policy Optimization in RLHF: Theoretical Insights on Efficient Data Utilizationcited by 25
  11. 2024RLeXplore: Accelerating Research in Intrinsically-Motivated Reinforcement Learningcited by 10
  12. 2024Intrinsic Rewards for Exploration without Harm from Observational Noise: A Simulation Study Based on the Free Energy Principlecited by 3
1 more
  1. 2026Curiosity-Critic: Cumulative Prediction Error Improvement as a Tractable Intrinsic Reward for World Model Trainingcited by 2

Built on by

Not cited yet.