mlsanityData to 14 Oct 2026

NBDI: A Simple and Effective Termination Condition for Skill Extraction from Task-Agnostic Demonstrations

arXiv 2501.12668 ↗First posted 22 Jan 2025cs.LG cs.AIcited by 1

Not on the map. Why?

Abstract (arXiv)

Intelligent agents are able to make decisions based on different levels of granularity and duration. Recent advances in skill learning enabled the agent to solve complex, long-horizon tasks by effectively guiding the agent in choosing appropriate skills. However, the practice of using fixed-length skills can easily result in skipping valuable decision points, which ultimately limits the potential for further exploration and faster policy learning. In this work, we propose to learn a simple and effective termination condition that identifies decision points through a state-action novelty module that leverages agent experience data. Our approach, Novelty-based Decision Point Identification (NBDI), outperforms previous baselines in complex, long-horizon tasks, and remains effective even in the presence of significant variations in the environment configurations of downstream tasks, highlighting the importance of decision point identification in skill learning.

Key facts. Cited by 1 paper in the map (data through 14 Oct 2026). Builds on 26 of its 43 references that are in the map. First posted on arXiv 22 Jan 2025.

Builds on

26 of 43 references.

References · 26

  1. 2018Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actorcited by 4,885
  2. 2021Offline Reinforcement Learning with Implicit Q-Learningcited by 1,293
  3. 2017Curiosity-driven Exploration by Self-supervised Predictioncited by 1,194
  4. 2018Exploration by Random Network Distillationcited by 993
  5. 2016Unifying Count-Based Exploration and Intrinsic Motivationcited by 639
  6. 2020Planning to Explore via Self-Supervised World Modelscited by 368
  7. 2016Variational Intrinsic Controlcited by 267
  8. 2019Self-Supervised Exploration via Disagreementcited by 265
  9. 2018Provably Efficient Maximum Entropy Explorationcited by 242
  10. 2020Accelerating Reinforcement Learning with Learned Skill Priorscited by 223
  11. 2020OPAL: Offline Primitive Discovery for Accelerating Offline Reinforcement Learningcited by 166
  12. 2020Parrot: Data-Driven Behavioral Priors for Reinforcement Learningcited by 138
14 more
  1. 2018Variational Option Discovery Algorithmscited by 122
  2. 2017A Laplacian Framework for Option Discovery in Reinforcement Learningcited by 110
  3. 2021State Entropy Maximization with Random Encoders for Efficient Explorationcited by 108
  4. 2017Eigenoption Discovery through the Deep Successor Representationcited by 86
  5. 2023METRA: Scalable Unsupervised RL with Metric-Aware Abstractioncited by 86
  6. 2021Demonstration-Guided Reinforcement Learning with Learned Skillscited by 76
  7. 2016Deep Successor Reinforcement Learningcited by 72
  8. 2021TRAIL: Near-Optimal Imitation Learning with Suboptimal Datacited by 39
  9. 2021Hierarchical Few-Shot Imitation with Skill Transition Modelscited by 37
  10. 2022Skill-based Meta-Reinforcement Learningcited by 37
  11. 2021How to Stay Curious while Avoiding Noisy TVs using Aleatoric Uncertainty Estimationcited by 27
  12. 2022Learning Options via Compressioncited by 22
  13. 2022MO2: Model-Based Offline Optionscited by 10
  14. 2016Option Discovery in Hierarchical Reinforcement Learning using Spatio-Temporal Clusteringcited by 6

Built on by

Cited by year

Citing papers per year: 2025: 1.

1 paper.

  1. 2025Beyond RAG vs. Long-Context: Learning Distraction-Aware Retrieval for Efficient Knowledge Groundingcited by 0