mlsanityData to 14 Oct 2026

Hierarchical Foresight: Self-Supervised Learning of Long-Horizon Tasks via Visual Subgoal Generation

arXiv 1909.05829 ↗First posted 12 Sep 2019cs.LG cs.AI cs.CVcited by 83

Not on the map. Why?

Abstract (arXiv)

Video prediction models combined with planning algorithms have shown promise in enabling robots to learn to perform many vision-based tasks through only self-supervision, reaching novel goals in cluttered scenes with unseen objects. However, due to the compounding uncertainty in long horizon video prediction and poor scalability of sampling-based planning optimizers, one significant limitation of these approaches is the ability to plan over long horizons to reach distant goals. To that end, we propose a framework for subgoal generation and planning, hierarchical visual foresight (HVF), which generates subgoal images conditioned on a goal image, and uses them for planning. The subgoal images are directly optimized to decompose the task into easy to plan segments, and as a result, we observe that the method naturally identifies semantically meaningful states as subgoals. Across three out of four simulated vision-based manipulation tasks, we find that our method achieves nearly a 200% performance improvement over planning without subgoals and model-free RL approaches. Further, our experiments illustrate that our approach extends to real, cluttered visual scenes. Project page: https://sites.google.com/stanford.edu/hvf

Key facts. Cited by 83 papers in the map (data through 14 Oct 2026). First posted on arXiv 12 Sep 2019.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2020: 1; 2021: 13; 2022: 17; 2023: 15; 2024: 16; 2025: 7; 2026: 14.

83 papers.

Sort:

By area:

  1. 2022Do As I Can, Not As I Say: Grounding Language in Robotic Affordancescited by 2,348
  2. 2022Inner Monologue: Embodied Reasoning through Planning with Language Modelscited by 1,081
  3. 2022ProgPrompt: Generating Situated Robot Task Plans using Large Language Modelscited by 611
  4. 2022Goal-Conditioned Reinforcement Learning: Problems and Solutionscited by 158
  5. 2022PONI: Potential Functions for ObjectGoal Navigation with Interaction-free Learningcited by 150
  6. 2026π0.7: a Steerable Generalist Robotic Foundation Model with Emergent Capabilitiescited by 138
  7. 2021Goal-Conditioned Reinforcement Learning with Imagined Subgoalscited by 134
  8. 2021Learning Language-Conditioned Robot Behavior from Offline Data and Crowd-Sourced Annotationcited by 118
  9. 2023Toward General-Purpose Robots via Foundation Models: A Survey and Meta-Analysiscited by 103
  10. 2021FitVid: Overfitting in Pixel-Level Video Predictioncited by 87
  11. 2020COG: Connecting New Skills to Past Experience with Offline Reinforcement Learningcited by 83
  12. 2022Latent Plans for Task-Agnostic Offline Reinforcement Learningcited by 81
38 more
  1. 2021Greedy Hierarchical Variational Autoencoders for Large-Scale Video Predictioncited by 75
  2. 2022Can Foundation Models Perform Zero-Shot Task Specification For Robot Manipulation?cited by 70
  3. 2022A Survey on Model-based Reinforcement Learningcited by 66
  4. 2024LoTa-Bench: Benchmarking Language-oriented Task Planners for Embodied Agentscited by 51
  5. 2021Landmark-Guided Subgoal Generation in Hierarchical Reinforcement Learningcited by 47
  6. 2025What Drives Success in Physical Planning with Joint-Embedding Predictive World Models?cited by 43
  7. 2023Contrastive Difference Predictive Codingcited by 36
  8. 2022LEAGUE: Guided Skill Learning and Abstraction for Long-Horizon Manipulationcited by 34
  9. 2022Planning to Practice: Efficient Online Fine-Tuning by Composing Goals in Latent Spacecited by 31
  10. 2021Model-Based Reinforcement Learning via Latent-Space Collocationcited by 28
  11. 2023Universal Visual Decomposer: Long-Horizon Manipulation Made Easycited by 25
  12. 2025Option-aware Temporally Abstracted Value for Offline Goal-Conditioned Reinforcement Learningcited by 23
  13. 2021Subgoal Search For Complex Reasoning Taskscited by 21
  14. 2022Generating Executable Action Plans with Environmentally-Aware Language Modelscited by 21
  15. 2021Solving Compositional Reinforcement Learning Problems via Task Reductioncited by 19
  16. 2021Active Hierarchical Exploration with Stable Subgoal Representation Learningcited by 17
  17. 2023Learning to Reason over Scene Graphs: A Case Study of Finetuning GPT-2 into a Robot Language Model for Grounded Task Planningcited by 16
  18. 2023Generalizable Long-Horizon Manipulations with Large Language Modelscited by 16
  19. 2023Hierarchical Imitation Learning with Vector Quantized Modelscited by 15
  20. 2024GHIL-Glue: Hierarchical Control with Filtered Subgoal Imagescited by 15
  21. 2025Learning Compositional Behaviors from Demonstration and Languagecited by 15
  22. 2024Scaling Robot Policy Learning via Zero-Shot Labeling with Foundation Modelscited by 14
  23. 2024Solving Minimum-Cost Reach Avoid using Reinforcement Learningcited by 14
  24. 2025Efficient Robotic Policy Learning via Latent Space Backward Planningcited by 14
  25. 2021Example-Driven Model-Based Reinforcement Learning for Solving Long-Horizon Visuomotor Taskscited by 13
  26. 2021Learning Domain Invariant Representations in Goal-conditioned Block MDPscited by 12
  27. 2022Fast and Precise: Adjusting Planning Horizon with Adaptive Subgoal Searchcited by 12
  28. 2022KEMP: Keyframe-Based Hierarchical End-to-End Deep Model for Long-Term Trajectory Predictioncited by 11
  29. 2022Discrete Factorial Representations as an Abstraction for Goal Conditioned Reinforcement Learningcited by 11
  30. 2026DynVLA: Learning World Dynamics for Action Reasoning in Autonomous Drivingcited by 11
  31. 2021LS3: Latent Space Safe Sets for Long-Horizon Visuomotor Control of Sparse Reward Iterative Taskscited by 10
  32. 2023Hieros: Hierarchical Imagination on Structured State Space Sequence World Modelscited by 8
  33. 2024Probabilistic Subgoal Representations for Hierarchical Reinforcement learningcited by 8
  34. 2024Pre-trained Visual Dynamics Representations for Efficient Policy Learningcited by 8
  35. 2026ForeAct: Steering Your VLA with Efficient Visual Foresight Planningcited by 8
  36. 2022Learning Sensorimotor Primitives of Sequential Manipulation Tasks from Visual Demonstrationscited by 7
  37. 2023Masked Diffusion with Task-awareness for Procedure Planning in Instructional Videoscited by 7
  38. 2025Structured Preference Optimization for Vision-Language Long-Horizon Task Planningcited by 7

50 most cited of 83 All 83 (JSON)