mlsanityData to 7 Oct 2026

Deeply AggreVaTeD: Differentiable Imitation Learning for Sequential Prediction

arXiv 1703.01030 ↗First posted 3 Mar 2017cs.LGcited by 93

Not on the map. Why?

Cited by 61 in Offline RL and imitation learning from data, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

Researchers have demonstrated state-of-the-art performance in sequential decision making problems (e.g., robotics control, sequential prediction) with deep neural network models. One often has access to near-optimal oracles that achieve good performance on the task during training. We demonstrate that AggreVaTeD --- a policy gradient extension of the Imitation Learning (IL) approach of (Ross & Bagnell, 2014) --- can leverage such an oracle to achieve faster and better solutions with less training data than a less-informed Reinforcement Learning (RL) technique. Using both feedforward and recurrent neural network predictors, we present stochastic gradient procedures on a sequential prediction task, dependency-parsing from raw image data, as well as on various high dimensional robotics control problems. We also provide a comprehensive theoretical study of IL that demonstrates we can expect up to exponentially lower sample complexity for learning with AggreVaTeD than with RL algorithms, which backs our empirical findings. Our results and theory indicate that the proposed approach can achieve superior performance with respect to the oracle when the demonstrator is sub-optimal.

Key facts. Cited by 93 papers in the map (data through 7 Oct 2026). First posted on arXiv 3 Mar 2017.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2018: 8; 2019: 9; 2020: 10; 2021: 12; 2022: 12; 2023: 17; 2024: 10; 2025: 11; 2026: 4.

93 papers.

Sort:

By area:

  1. 2018An Algorithmic Perspective on Imitation Learningcited by 455
  2. 2024Rewarding Progress: Scaling Automated Process Verifiers for LLM Reasoningcited by 267
  3. 2023A Survey of Imitation Learning: Algorithms, Recent Developments, and Challengescited by 145
  4. 2020What are the Statistical Limits of Offline RL with Linear Function Approximation?cited by 135
  5. 2021Urban Driver: Learning to Drive from Real-world Demonstrations Using Policy Gradientscited by 122
  6. 2019Imitation Learning as f-Divergence Minimizationcited by 121
  7. 2022Adversarially Trained Actor Critic for Offline Reinforcement Learningcited by 111
  8. 2019Help, Anna! Visual Navigation with Natural Multimodal Assistance via Retrospective Curiosity-Encouraging Imitation Learningcited by 100
  9. 2020Toward the Fundamental Limits of Imitation Learningcited by 96
  10. 2019BAIL: Best-Action Imitation Learning for Batch Deep Reinforcement Learningcited by 90
  11. 2019Provably Efficient Imitation Learning from Observation Alonecited by 78
  12. 2018Vision-based Navigation with Language-based Assistance via Imitation Learning with Indirect Interventioncited by 77
38 more
  1. 2020Fighting Copycat Agents in Behavioral Cloning from Observation Historiescited by 70
  2. 2020Autoregressive Knowledge Distillation through Imitation Learningcited by 68
  3. 2021Imitation Learning: Progress, Taxonomies and Challengescited by 56
  4. 2023Learning to Generate Better Than Your LLMcited by 49
  5. 2019Random Expert Distillation: Imitation Learning via Expert Policy Support Estimationcited by 42
  6. 2021Heuristic-Guided Reinforcement Learningcited by 41
  7. 2018Truncated Horizon Policy Search: Combining Reinforcement Learning & Imitation Learningcited by 39
  8. 2021On the Sample Complexity of Stability Constrained Imitation Learningcited by 38
  9. 2020Imitation Learning with Stability and Safety Guaranteescited by 34
  10. 2025Multi-Turn Code Generation Through Single-Step Rewardscited by 27
  11. 2021Mitigating Covariate Shift in Imitation Learning via Offline Data Without Great Coveragecited by 26
  12. 2023CCIL: Continuity-based Data Augmentation for Corrective Imitation Learningcited by 26
  13. 2021Keyframe-Focused Visual Imitation Learningcited by 25
  14. 2022Hindsight Learning for MDPs with Exogenous Inputscited by 25
  15. 2023RLIF: Interactive Imitation Learning as Reinforcement Learningcited by 24
  16. 2020Scalable Multi-Task Imitation Learning with Autonomous Improvementcited by 21
  17. 2023Chain-of-Thought Predictive Controlcited by 18
  18. 2021Object-Aware Regularization for Addressing Causal Confusion in Imitation Learningcited by 16
  19. 2023Blending Imitation and Reinforcement Learning for Robust Policy Improvementcited by 16
  20. 2024Diffusing States and Matching Scores: A New Framework for Imitation Learningcited by 16
  21. 2022POETREE: Interpretable Policy Learning with Adaptive Decision Treescited by 14
  22. 2022Resolving Copycat Problems in Visual Imitation Learning via Residual Action Predictioncited by 14
  23. 2025Qsharp: Provably Optimal Distributional RL for LLM Post-Trainingcited by 14
  24. 2020Synthesizing Decentralized Controllers with Graph Neural Networks and Imitation Learningcited by 12
  25. 2021Provably Breaking the Quadratic Error Compounding Barrier in Imitation Learning, Optimallycited by 12
  26. 2018OIL: Observational Imitation Learningcited by 11
  27. 2023Active Policy Improvement from Multiple Black-box Oraclescited by 11
  28. 2024MaIL: Improving Imitation Learning with Mambacited by 11
  29. 2019On-Policy Robot Imitation Learning from a Converging Supervisorcited by 10
  30. 2023R-U-SURE? Uncertainty-Aware Code Suggestions By Maximizing Utility Across Random User Intentscited by 10
  31. 2023Guarded Policy Optimization with Imperfect Online Demonstrationscited by 10
  32. 2026Good SFT Optimizes for SFT, Better SFT Prepares for Reinforcement Learningcited by 10
  33. 2023How To Guide Your Learner: Imitation Learning with Active Adaptive Expert Involvementcited by 8
  34. 2023Adaptive action supervision in reinforcement learning from real-world multi-agent demonstrationscited by 8
  35. 2024Control-oriented Clustering of Visual Latent Representationcited by 8
  36. 2018Convergence of Value Aggregation for Imitation Learningcited by 7
  37. 2019Integration of Imitation Learning using GAIL and Reinforcement Learning using Task-achievement Rewards via Probabilistic Graphical Modelcited by 7
  38. 2022A Few Expert Queries Suffices for Sample-Efficient RL with Resets and Linear Value Approximationcited by 7

50 most cited of 93 All 93 (JSON)