mlsanityData to 14 Oct 2026

Autonomous Learning From Success and Failure: Goal-Conditioned Supervised Learning with Negative Feedback

arXiv 2509.03206 ↗First posted 3 Sep 2025cs.LG cs.AIcited by 0

On the map: Offline RL and imitation learning from data → Imitation learning methods

Abstract (arXiv)

Learning from reward functions and imitation learning of demonstrations are the two principal approaches for training autonomous systems that interact with an environment through action and observation. Both, however, require human specification for each behaviour to be acquired, a problem for long-lived self-adaptive systems whose goals and operating conditions cannot be fully anticipated at design time. Recently, Goal-Conditioned Supervised Learning (GCSL) through self-imitation has been proposed as a self-supervised alternative: by strategically relabelling goals, agents can derive policy insights from their own experiences. Despite its successes, this framework presents two notable limitations: (1) learning exclusively from self-generated experiences can exacerbate the agents' inherent biases; (2) the relabelling strategy allows agents to focus solely on successful outcomes, precluding them from learning from their mistakes. To address these issues, we propose GCSL with Negative Feedback (GCSL-NF), which evaluates each trajectory twice: positively with respect to relabelled goals, and correctively with respect to the goal originally intended. The corrective target comes from a similarity function learned contrastively from trajectory-induced neighbourhood relations, so that neither a reward function nor a geometric distance needs to be specified. Our experiments show that GCSL-NF overcomes limitations imposed by agents' initial biases, increasingly benefits from negative feedback as learning progresses, and matches or surpasses GCSL- and HER-based methods. By reducing reliance on prespecified reward functions, the proposed approach is particularly relevant for self-adaptive autonomous systems, where adaptation objectives may be diverse, changing, or difficult to engineer.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in Imitation learning methods, one of 13 directions in Offline RL and imitation learning from data. Builds on 51 of its 84 references that are in the map. First posted on arXiv 3 Sep 2025.

Builds on

51 of 84 references.

Closest ancestors · 4

Same direction, newest first

  1. 2022Imitating Past Successes can be Very Suboptimal7 Jun
  2. 2019Independent Generative Adversarial Self-Imitation Learning in Cooperative Multiagent Systems25 Sep
  3. 2019Goal-conditioned Imitation Learning13 Jun
  4. 2018Self-Imitation Learning14 Jun

Other references · 47

  1. 2020A Simple Framework for Contrastive Learning of Visual Representationscited by 12,049
  2. 2019Momentum Contrast for Unsupervised Visual Representation Learningcited by 7,513
  3. 2013Distributed Representations of Words and Phrases and their Compositionalitycited by 3,474
  4. 2015Continuous control with deep reinforcement learningcited by 3,407
  5. 2020Unsupervised Learning of Visual Features by Contrasting Cluster Assignmentscited by 2,857
  6. 2016Asynchronous Methods for Deep Reinforcement Learningcited by 2,258
  7. 2015Deep Reinforcement Learning with Double Q-learningcited by 1,745
  8. 2020Big Self-Supervised Models are Strong Semi-Supervised Learnerscited by 1,380
  9. 2017Curiosity-driven Exploration by Self-supervised Predictioncited by 1,194
  10. 2017Hindsight Experience Replaycited by 1,136
  11. 2016End to End Learning for Self-Driving Carscited by 1,009
  12. 2018Exploration by Random Network Distillationcited by 993
35 more
  1. 2017Constrained Policy Optimizationcited by 983
  2. 2019Advantage-Weighted Regression: Simple and Scalable Off-Policy Reinforcement Learningcited by 782
  3. 2020CURL: Contrastive Unsupervised Representations for Reinforcement Learningcited by 760
  4. 2017Safe Reinforcement Learning via Shieldingcited by 449
  5. 2018Reward Constrained Policy Optimizationcited by 417
  6. 2019Relay Policy Learning: Solving Long-Horizon Tasks via Imitation and Reinforcement Learningcited by 384
  7. 2018Maximum a Posteriori Policy Optimisationcited by 328
  8. 2017Leveraging Demonstrations for Deep Reinforcement Learning on Robotics Problems with Sparse Rewardscited by 319
  9. 2019Learning Latent Plans from Playcited by 316
  10. 2019End-to-End Safe Reinforcement Learning through Barrier Functions for Safety-Critical Continuous Control Taskscited by 289
  11. 2020Decoupling Representation Learning from Reinforcement Learningcited by 269
  12. 2018Multi-Goal Reinforcement Learning: Challenging Robotics Environments and Request for Researchcited by 260
  13. 2022Contrastive Learning as Goal-Conditioned Reinforcement Learningcited by 253
  14. 2020InfoXLM: An Information-Theoretic Framework for Cross-Lingual Language Model Pre-Trainingcited by 237
  15. 2020Recovery RL: Safe Reinforcement Learning with Learned Recovery Zonescited by 175
  16. 2020A Closer Look at Invalid Action Masking in Policy Gradient Algorithmscited by 171
  17. 2022Exploration in Deep Reinforcement Learning: A Surveycited by 166
  18. 2015Incentivizing Exploration In Reinforcement Learning With Deep Predictive Modelscited by 154
  19. 2019Planning with Goal-Conditioned Policiescited by 149
  20. 2021Goal-Conditioned Reinforcement Learning with Imagined Subgoalscited by 134
  21. 2018Noise Contrastive Estimation and Negative Sampling for Conditional Models: Consistency and Statistical Efficiencycited by 105
  22. 2017Trial without Error: Towards Safe Reinforcement Learning via Human Interventioncited by 103
  23. 2022Self-Supervised Learning for Recommender Systems: A Surveycited by 95
  24. 2022Rethinking Goal-conditioned Supervised Learning and Its Connection to Offline RLcited by 89
  25. 2017Intrinsically Motivated Goal Exploration Processes with Automatic Curriculum Learningcited by 83
  26. 2016Deep Successor Reinforcement Learningcited by 72
  27. 2017Hindsight policy gradientscited by 37
  28. 2019Self-supervised Learning of Distance Functions for Goal-Conditioned Reinforcement Learningcited by 18
  29. 2024TLDR: Unsupervised Goal-Conditioned RL via Temporal Distance-Aware Representationscited by 17
  30. 2021SimSR: Simple Distance-based State Representation for Deep Reinforcement Learningcited by 14
  31. 2022State Representation Learning for Goal-Conditioned Reinforcement Learningcited by 11
  32. 2022LCRL: Certified Policy Synthesis via Logically-Constrained Reinforcement Learningcited by 11
  33. 2019Feature-Model-Guided Online Learning for Self-Adaptive Systemscited by 7
  34. 2021Uncertainty in Self-Adaptive Systems: A Research Community Perspectivecited by 6
  35. 2022Explaining Online Reinforcement Learning Decisions of Self-Adaptive Systemscited by 5

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2019Learning to Reach Goals via Iterated Supervised Learning
  2. 2025Consistent Zero-Shot Imitation with Contrastive Goal Inference
  3. 2026Unifying Goal-Conditioned RL and Unsupervised Skill Learning via Control-Maximization
  4. 2019Reward-Conditioned Policies
  5. 2022Understanding Hindsight Goal Relabeling from a Divergence Minimization Perspective
  6. 2023Backward Learning for Goal-Conditioned Policies
  7. 2019From Language to Goals: Inverse Reinforcement Learning for Vision-Based Instruction Following
  8. 2026Learning from Demonstrations via Capability-Aware Goal Sampling