mlsanityData to 7 Oct 2026

Truly Batch Apprenticeship Learning with Deep Successor Features

arXiv 1903.10077 ↗First posted 24 Mar 2019cs.LG stat.MLcited by 27

Not on the map. Why?

Abstract (arXiv)

We introduce a novel apprenticeship learning algorithm to learn an expert's underlying reward structure in off-policy model-free batch settings. Unlike existing methods that require a dynamics model or additional data acquisition for on-policy evaluation, our algorithm requires only the batch data of observed expert behavior. Such settings are common in real-world tasks---health care, finance or industrial processes ---where accurate simulators do not exist or data acquisition is costly. To address challenges in batch settings, we introduce Deep Successor Feature Networks(DSFN) that estimate feature expectations in an off-policy setting and a transition-regularized imitation network that produces a near-expert initial policy and an efficient feature representation. Our algorithm achieves superior results in batch settings on both control benchmarks and a vital clinical task of sepsis management in the Intensive Care Unit.

Key facts. Cited by 27 papers in the map (data through 7 Oct 2026). First posted on arXiv 24 Mar 2019.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2019: 1; 2020: 1; 2021: 7; 2022: 3; 2023: 6; 2024: 5; 2025: 1; 2026: 3.

27 papers.

Sort:

By area:

  1. 2021IQ-Learn: Inverse soft-Q Learning for Imitationcited by 191
  2. 2021Scalable Bayesian Inverse Reinforcement Learningcited by 56
  3. 2023Benchmarks and Algorithms for Offline Preference-Based Reward Learningcited by 55
  4. 2020Strictly Batch Imitation Learning by Energy-based Distribution Matchingcited by 54
  5. 2023When Demonstrations Meet Generative World Models: A Maximum Likelihood Framework for Offline Inverse Reinforcement Learningcited by 26
  6. 2021PsiPhi-Learning: Reinforcement Learning with Demonstrations using Successor Features and Inverse Temporal Difference Learningcited by 23
  7. 2023CLARE: Conservative Model-Based Reward Learning for Offline Inverse Reinforcement Learningcited by 20
  8. 2022Benchmarking Constraint Inference in Inverse Reinforcement Learningcited by 18
  9. 2021Offline Preference-Based Apprenticeship Learningcited by 17
  10. 2022Dynamics-Aware Comparison of Learned Reward Functionscited by 17
  11. 2019Inverse Reinforcement Learning in Contextual MDPscited by 12
  12. 2021Inverse Contextual Bandits: Learning How Behavior Evolves over Timecited by 12
15 more
  1. 2024How to Leverage Diverse Demonstrations in Offline Imitation Learningcited by 11
  2. 2023Inverse Decision Modeling: Learning Interpretable Representations of Behaviorcited by 10
  3. 2024Non-Adversarial Inverse Reinforcement Learning via Successor Feature Matchingcited by 10
  4. 2021Successor Feature Sets: Generalizing Successor Representations Across Policiescited by 6
  5. 2022Basis for Intentions: Efficient Inverse Reinforcement Learning using Past Experiencecited by 4
  6. 2024Predictive representations: building blocks of intelligencecited by 4
  7. 2024Markov Balance Satisfaction Improves Performance in Strictly Batch Offline Imitation Learningcited by 3
  8. 2023Conditional Kernel Imitation Learning for Continuous State Environmentscited by 2
  9. 2025Language Model Distillation: A Temporal Difference Imitation Learning Perspectivecited by 2
  10. 2026A Generalized Apprenticeship Learning Framework for Capturing Evolving Student Pedagogical Strategiescited by 2
  11. 2023Replicating Complex Dialogue Policy of Humans via Offline Imitation Learning with Supervised Regularizationcited by 1
  12. 2024OMG-RL:Offline Model-based Guided Reward Learning for Heparin Treatmentcited by 1
  13. 2021Extracting Expert's Goals by What-if Interpretable Modelingcited by 0
  14. 2026A Context Engineering Framework for Improving Enterprise AI Agents based on Digital-Twin MDPcited by 0
  15. 2026Quantifying Potential Observation Missingness in Inverse Reinforcement Learningcited by 0