mlsanityData to 7 Oct 2026

Recent Advances in Leveraging Human Guidance for Sequential Decision-Making Tasks

arXiv 2107.05825 ↗First posted 13 Jul 2021cs.AI cs.LGcited by 13

Not on the map. Why?

Abstract (arXiv)

A longstanding goal of artificial intelligence is to create artificial agents capable of learning to perform tasks that require sequential decision making. Importantly, while it is the artificial agent that learns and acts, it is still up to humans to specify the particular task to be performed. Classical task-specification approaches typically involve humans providing stationary reward functions or explicit demonstrations of the desired tasks. However, there has recently been a great deal of research energy invested in exploring alternative ways in which humans may guide learning agents that may, e.g., be more suitable for certain tasks or require less human effort. This survey provides a high-level overview of five recent machine learning frameworks that primarily rely on human guidance apart from pre-specified reward functions or conventional, step-by-step action demonstrations. We review the motivation, assumptions, and implementation of each framework, and we discuss possible future research directions.

Key facts. Cited by 13 papers in the map (data through 7 Oct 2026). Builds on 108 of its 232 references that are in the map. First posted on arXiv 13 Jul 2021.

Builds on

108 of 232 references.

References · 108

  1. 2017Deep reinforcement learning from human preferencescited by 3,846
  2. 2015Trust Region Policy Optimizationcited by 2,503
  3. 2016Generative Adversarial Imitation Learningcited by 1,576
  4. 2016Realtime Multi-Person 2D Pose Estimation using Part Affinity Fieldscited by 1,061
  5. 2016End to End Learning for Self-Driving Carscited by 1,000
  6. 2015End-to-End Training of Deep Visuomotor Policiescited by 847
  7. 2015Dueling Network Architectures for Deep Reinforcement Learningcited by 769
  8. 2017Deep Reinforcement Learning that Matterscited by 727
  9. 2017Unpaired Image-to-Image Translation using Cycle-Consistent Adversarial Networkscited by 707
  10. 2018Deep Visual Domain Adaptation: A Surveycited by 634
  11. 2017Learning Robust Rewards with Adversarial Inverse Reinforcement Learningcited by 485
  12. 2018Behavioral Cloning from Observationcited by 482
38 more
  1. 2017End-to-end Driving via Conditional Imitation Learningcited by 479
  2. 2018An Algorithmic Perspective on Imitation Learningcited by 455
  3. 2017Time-Contrastive Networks: Self-Supervised Learning from Videocited by 453
  4. 2018Data-Efficient Hierarchical Reinforcement Learningcited by 436
  5. 2016The Option-Critic Architecturecited by 433
  6. 2016Guided Cost Learning: Deep Inverse Optimal Control via Policy Optimizationcited by 400
  7. 2019Relay Policy Learning: Solving Long-Horizon Tasks via Imitation and Reinforcement Learningcited by 380
  8. 2017FeUdal Networks for Hierarchical Reinforcement Learningcited by 328
  9. 2018Reward learning from human preferences and demonstrations in Ataricited by 314
  10. 2016Cooperative Inverse Reinforcement Learningcited by 298
  11. 2018A Survey of Inverse Reinforcement Learning: Challenges, Methods and Progresscited by 294
  12. 2016Learning from Simulated and Unsupervised Images through Adversarial Trainingcited by 287
  13. 2016Hierarchical Deep Reinforcement Learning: Integrating Temporal Abstraction and Intrinsic Motivationcited by 282
  14. 2017Revisiting the Arcade Learning Environment: Evaluation Protocols and Open Problems for General Agentscited by 267
  15. 2018Human-level performance in first-person multiplayer games with population-based deep reinforcement learningcited by 260
  16. 2019Extrapolating Beyond Suboptimal Demonstrations via Inverse Reinforcement Learning from Observationscited by 256
  17. 2018Discriminator-Actor-Critic: Addressing Sample Inefficiency and Reward Bias in Adversarial Imitation Learningcited by 203
  18. 2014The Secrets of Salient Object Segmentationcited by 200
  19. 2016Modular Multitask Reinforcement Learning with Policy Sketchescited by 200
  20. 2016What do different evaluation metrics tell us about saliency models?cited by 184
  21. 2018Generative Adversarial Imitation from Observationcited by 183
  22. 2018A Survey and Critique of Multiagent Deep Reinforcement Learningcited by 173
  23. 2017Imitation from Observation: Learning to Imitate Behaviors from Raw Video via Context Translationcited by 167
  24. 2016Eye Tracking for Everyonecited by 155
  25. 2017Interactive Learning from Policy-Dependent Human Feedbackcited by 155
  26. 2015Appearance-Based Gaze Estimation in the Wildcited by 153
  27. 2018DeepMimic: Example-Guided Deep Reinforcement Learning of Physics-Based Character Skillscited by 153
  28. 2015Salient Object Detection: A Benchmarkcited by 130
  29. 2017Deep TAMER: Interactive Agent Shaping in High-Dimensional State Spacescited by 122
  30. 2017MPIIGaze: Real-World Dataset and Deep Appearance-Based Gaze Estimationcited by 118
  31. 2018Zero-Shot Visual Imitationcited by 114
  32. 2017Third-Person Imitation Learningcited by 103
  33. 2017Trial without Error: Towards Safe Reinforcement Learning via Human Interventioncited by 102
  34. 2018Imitating Latent Policies from Observationcited by 101
  35. 2018Playing hard exploration games by watching YouTubecited by 98
  36. 2019Recent Advances in Imitation Learning from Observationcited by 92
  37. 2020Deep Visual Domain Adaptationcited by 91
  38. 2017Combining Self-Supervised Learning and Imitation for Vision-Based Rope Manipulationcited by 90

50 most cited of 108 All 108 (JSON)

Built on by

Cited by year

Citing papers per year: 2021: 1; 2022: 5; 2023: 3; 2024: 2; 2025: 2.

13 papers.

Sort:

By area:

  1. 2022Efficient Learning of Safe Driving Policy via Human-AI Copilot Optimizationcited by 44
  2. 2022Improving Multimodal Interactive Agents with Reinforcement Learning from Human Feedbackcited by 26
  3. 2023Accelerating Reinforcement Learning of Robotic Manipulations via Feedback from Large Language Modelscited by 22
  4. 2021Beyond Robustness: A Taxonomy of Approaches towards Resilient Multi-Robot Systemscited by 12
  5. 2023NOIR: Neural Signal Operated Intelligent Robots for Everyday Activitiescited by 8
  6. 2022Reinforcement Learning in Practice: Opportunities and Challengescited by 5
  7. 2022Human-AI Shared Control via Policy Dissectioncited by 4
  8. 2025Human-Aligned Skill Discovery: Balancing Behaviour Exploration and Alignmentcited by 3
  9. 2022Human in the loop approaches in multi-modal conversational task guidance system developmentcited by 2
  10. 2023CUEING: a lightweight model to Capture hUman attEntion In driviNGcited by 1
  11. 2024Aligning Agents like Large Language Modelscited by 1
  12. 2024Prosody as a Teaching Signal for Agent Learning: Exploratory Studies and Algorithmic Implicationscited by 0
1 more
  1. 2025NatSGLD: A Dataset with Speech, Gesture, Logic, and Demonstration for Robot Learning in Natural Human-Robot Interactioncited by 0