mlsanityData to 14 Oct 2026

EvoHIL: Self-Evolving Reward and Flow-Matched Policy Optimization for Robust Human-in-the-Loop Reinforcement Learning

arXiv 2608.03872 ↗First posted 4 Aug 2026cs.ROcited by 0

On the map: Deep reinforcement learning methods → RL for robot locomotion and manipulation

Abstract (arXiv)

Human-in-the-loop reinforcement learning (HIL-RL) enables robots to learn contact-rich manipulation from limited real-world interaction, but deployment exposes three coupled limitations: static visual reward models fail under scene changes; independently sampled actions cause temporally inconsistent motion; and vision-based policies remain sensitive to appearance shifts. We present EvoHIL, a unified framework that adapts the reward model, action generator, and visual do main within a staged human-in-the-loop learning process. First, self-evolving reward (SER) adapts the success classifier from human-confirmed positives and provisional weak negatives. Second, Action Flow Stabilization (AFS) generates temporally coherent action chunks through flow matching, grounding policy updates in executed action prefixes and demonstrated behavior. Third, retention-aware offline fine-tuning replays relit interaction data while anchoring the AFS actor-critic to prior behavior, adapting the visual domain without additional robot interaction. Across six manipulation tasks on Franka FR3 and SO-101 arms under a controlled lighting shift, EvoHIL improves task success, agreement with human-confirmation labels, motion smoothness, and completion time relative to human-in-the-loop and imitation baselines.Project page: https://anonymous4366.github.io/EvoHIL/

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in RL for robot locomotion and manipulation, one of 15 directions in Deep reinforcement learning methods. Builds on 33 of its 40 references that are in the map. First posted on arXiv 4 Aug 2026.

Builds on

33 of 40 references.

Closest ancestors · 2

Same direction, newest first

  1. 2024Precise and Dexterous Robotic Manipulation via Human-in-the-Loop Reinforcement Learning29 Oct
  2. 2024SERL: A Software Suite for Sample-Efficient Robotic Reinforcement Learning29 Jan

Other references · 31

  1. 2015Deep Residual Learning for Image Recognitioncited by 52,096
  2. 2022Flow Matching for Generative Modelingcited by 5,702
  3. 2017On Calibration of Modern Neural Networkscited by 4,960
  4. 2018Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actorcited by 4,885
  5. 2017Deep reinforcement learning from human preferencescited by 3,912
  6. 2022Flow Straight and Fast: Learning to Generate and Transfer Data with Rectified Flowcited by 3,700
  7. 2023Diffusion Policy: Visuomotor Policy Learning via Action Diffusioncited by 2,838
  8. 2018Addressing Function Approximation Error in Actor-Critic Methodscited by 2,417
  9. 2023Learning Fine-Grained Bimanual Manipulation with Low-Cost Hardwarecited by 1,568
  10. 2017Domain Randomization for Transferring Deep Neural Networks from Simulation to the Real Worldcited by 1,498
  11. 2021Offline Reinforcement Learning with Implicit Q-Learningcited by 1,293
  12. 2019Can You Trust Your Model's Uncertainty? Evaluating Predictive Uncertainty Under Dataset Shiftcited by 1,287
19 more
  1. 2018Experience Replay for Continual Learningcited by 1,095
  2. 2019AugMix: A Simple Data Processing Method to Improve Robustness and Uncertaintycited by 901
  3. 2018QT-Opt: Scalable Deep Reinforcement Learning for Vision-Based Robotic Manipulationcited by 743
  4. 2020Image Augmentation Is All You Need: Regularizing Deep Reinforcement Learning from Pixelscited by 585
  5. 2020Reinforcement Learning with Augmented Datacited by 468
  6. 2021Implicit Behavioral Cloningcited by 452
  7. 2023Efficient Online Reinforcement Learning with Offline Datacited by 363
  8. 2024Diffusion Policy Policy Optimizationcited by 280
  9. 2018HG-DAgger: Interactive Imitation Learning with Human Expertscited by 266
  10. 2021How to Train Your Robot with Deep Reinforcement Learning; Lessons We've Learnedcited by 213
  11. 2025Reinforcement Learning with Action Chunkingcited by 107
  12. 2025Cosmos-Transfer1: Conditional World Generation with Adaptive Multimodal Controlcited by 106
  13. 2025Flow Matching Policy Gradientscited by 99
  14. 2025ReinFlow: Fine-tuning Flow Matching Policy with Online Reinforcement Learningcited by 97
  15. 2018Variational Inverse Control with Events: A General Framework for Data-Driven Reward Definitioncited by 69
  16. 2020Regularizing Action Policies for Smooth Control with Reinforcement Learningcited by 59
  17. 2025UniRelight: Learning Joint Decomposition and Synthesis for Video Relightingcited by 40
  18. 2019Positive-Unlabeled Reward Learningcited by 37
  19. 2025SAC Flow: Sample-Efficient Reinforcement Learning of Flow-Based Policies via Velocity-Reparameterized Sequential Modelingcited by 31

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2026Preference-Calibrated Human-in-the-Loop Reinforcement Learning for Robotic Manipulation
  2. 2026ARM: Advantage Reward Modeling for Long-Horizon Manipulation
  3. 2026Res-HIL: Human-Guided Residual Reinforcement Learning for Sample-Efficient Dexterous Manipulation
  4. 2026FlowDAgger: Human-in-the-Loop Adaptation of Generative Robot Policies in Latent Space
  5. 2026SARM2: Multi-Task Stage Aware Reward Modeling for Self Improving Robotic Manipulation
  6. 2026Large Reward Models: Generalizable Online Robot Reward Generation with Vision-Language Models
  7. 2026RoMAN-Flow: Taming Autoregressive Normalizing Flows for Offline Reinforcement Learning in Robotic Manipulation
  8. 2026ReF-HIL: Shaping the Critic around Human Action Neighborhoods for Efficient Human-in-the-Loop Reinforcement Learning