mlsanityData to 14 Oct 2026

Noise-conditioned Energy-based Annealed Rewards (NEAR): A Generative Framework for Imitation Learning from Observation

arXiv 2501.14856 ↗First posted 24 Jan 2025cs.RO cs.AIcited by 3

On the map: Offline RL and imitation learning from data → Imitation learning methods

Abstract (arXiv)

This paper introduces a new imitation learning framework based on energy-based generative models capable of learning complex, physics-dependent, robot motion policies through state-only expert motion trajectories. Our algorithm, called Noise-conditioned Energy-based Annealed Rewards (NEAR), constructs several perturbed versions of the expert's motion data distribution and learns smooth, and well-defined representations of the data distribution's energy function using denoising score matching. We propose to use these learnt energy functions as reward functions to learn imitation policies via reinforcement learning. We also present a strategy to gradually switch between the learnt energy functions, ensuring that the learnt rewards are always well-defined in the manifold of policy-generated samples. We evaluate our algorithm on complex humanoid tasks such as locomotion and martial arts and compare it with state-only adversarial imitation learning algorithms like Adversarial Motion Priors (AMP). Our framework sidesteps the optimisation challenges of adversarial imitation learning techniques and produces results comparable to AMP in several quantitative metrics across multiple imitation settings.

Key facts. Cited by 3 papers in the map (data through 14 Oct 2026). Placed in Imitation learning methods, one of 13 directions in Offline RL and imitation learning from data. Builds on 16 of its 37 references that are in the map. First posted on arXiv 24 Jan 2025.

Builds on

16 of 37 references.

Closest ancestors · 6

Same direction, newest first

  1. 2024Diffusion Imitation from Observation7 Oct
  2. 2023DiffAIL: Diffusion Adversarial Imitation Learning11 Dec
  3. 2023A Survey of Imitation Learning: Algorithms, Recent Developments, and Challenges5 Sep
  4. 2019Recent Advances in Imitation Learning from Observation30 May
  5. 2018An Algorithmic Perspective on Imitation Learning16 Nov
  6. 2018Generative Adversarial Imitation from Observation17 Jul

Other references · 10

  1. 2017Proximal Policy Optimization Algorithmscited by 15,215
  2. 2019Generative Modeling by Estimating Gradients of the Data Distributioncited by 4,101
  3. 2023Diffusion Policy: Visuomotor Policy Learning via Action Diffusioncited by 2,838
  4. 2015High-Dimensional Continuous Control Using Generalized Advantage Estimationcited by 2,026
  5. 2016Generative Adversarial Imitation Learningcited by 1,597
  6. 2020Improved Techniques for Training Score-Based Generative Modelscited by 1,043
  7. 2021Isaac Gym: High Performance GPU-Based Physics Simulation For Robot Learningcited by 998
  8. 2017Towards Principled Methods for Training Generative Adversarial Networkscited by 464
  9. 2021How to Train Your Energy-Based Modelscited by 274
  10. 2023CALM: Conditional Adversarial Latent Models for Directable Virtual Characterscited by 94

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 3.

3 papers.

  1. 2026Recovering Hidden Reward in Diffusion-Based Policiescited by 0
  2. 2026Trust Region Inverse Reinforcement Learning: Explicit Dual Ascent using Local Policy Updatescited by 0
  3. 2026A Geometric Perspective on Reward Function Updates in Inverse Reinforcement Learningcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2020Energy-Based Imitation Learning
  2. 2023Revisiting Energy Based Models as Policies: Ranking Noise Contrastive Estimation and Interpolating Energy Models
  3. 2020On Computation and Generalization of Generative Adversarial Imitation Learning
  4. 2020Shaping Rewards for Reinforcement Learning with Imperfect Demonstrations using Generative Models
  5. 2019Random Expert Distillation: Imitation Learning via Expert Policy Support Estimation
  6. 2019A New Framework for Query Efficient Active Imitation Learning
  7. 2022Latent Policies for Adversarial Imitation Learning
  8. 2024Diffusion-Reward Adversarial Imitation Learning