# STAP: Sequencing Task-Agnostic Policies

arXiv: https://arxiv.org/abs/2210.12250 · HTML: https://mlsanity.com/paper/2210.12250/

## Key facts

- Cited by 18 papers in the map (data through 14 Oct 2026).
- Builds on 47 of its 83 references that are in the map.
- First posted on arXiv 21 Oct 2022.

## Builds on (most cited first, 30 of 47)

- [Dropout as a Bayesian Approximation: Representing Model Uncertainty in Deep Learning](https://mlsanity.com/paper/1506.02142/) (2015)
- [Continuous control with deep reinforcement learning](https://mlsanity.com/paper/1509.02971/) (2015)
- [Addressing Function Approximation Error in Actor-Critic Methods](https://mlsanity.com/paper/1802.09477/) (2018)
- [Dream to Control: Learning Behaviors by Latent Imagination](https://mlsanity.com/paper/1912.01603/) (2019)
- [Learning Latent Dynamics for Planning from Pixels](https://mlsanity.com/paper/1811.04551/) (2018)
- [Soft Actor-Critic Algorithms and Applications](https://mlsanity.com/paper/1812.05905/) (2018)
- [Deep Reinforcement Learning in a Handful of Trials using Probabilistic Dynamics Models](https://mlsanity.com/paper/1805.12114/) (2018)
- [When to Trust Your Model: Model-Based Policy Optimization](https://mlsanity.com/paper/1906.08253/) (2019)
- [QT-Opt: Scalable Deep Reinforcement Learning for Vision-Based Robotic Manipulation](https://mlsanity.com/paper/1806.10293/) (2018)
- [Learning Complex Dexterous Manipulation with Deep Reinforcement Learning and Demonstrations](https://mlsanity.com/paper/1709.10087/) (2017)
- [R3M: A Universal Visual Representation for Robot Manipulation](https://mlsanity.com/paper/2203.12601/) (2022)
- [Data-Efficient Hierarchical Reinforcement Learning](https://mlsanity.com/paper/1805.08296/) (2018)
- [The Option-Critic Architecture](https://mlsanity.com/paper/1609.05140/) (2016)
- [Planning to Explore via Self-Supervised World Models](https://mlsanity.com/paper/2005.05960/) (2020)
- [Dynamics-Aware Unsupervised Discovery of Skills](https://mlsanity.com/paper/1907.01657/) (2019)
- [Integrated Task and Motion Planning](https://mlsanity.com/paper/2010.01083/) (2020)
- [Deep Visual Foresight for Planning Robot Motion](https://mlsanity.com/paper/1610.00696/) (2016)
- [Ensemble deep learning: A review](https://mlsanity.com/paper/2104.02395/) (2021)
- [MT-Opt: Continuous Multi-Task Robotic Reinforcement Learning at Scale](https://mlsanity.com/paper/2104.08212/) (2021)
- [Accelerating Reinforcement Learning with Learned Skill Priors](https://mlsanity.com/paper/2010.11944/) (2020)
- [OPAL: Offline Primitive Discovery for Accelerating Offline Reinforcement Learning](https://mlsanity.com/paper/2010.13611/) (2020)
- [Parrot: Data-Driven Behavioral Priors for Reinforcement Learning](https://mlsanity.com/paper/2011.10024/) (2020)
- [Actionable Models: Unsupervised Offline Reinforcement Learning of Robotic Skills](https://mlsanity.com/paper/2104.07749/) (2021)
- [Augmenting Reinforcement Learning with Behavior Primitives for Diverse Manipulation Tasks](https://mlsanity.com/paper/2110.03655/) (2021)
- [Neural Task Programming: Learning to Generalize Across Hierarchical Tasks](https://mlsanity.com/paper/1710.01813/) (2017)
- [Greedy Hierarchical Variational Autoencoders for Large-Scale Video Prediction](https://mlsanity.com/paper/2103.04174/) (2021)
- [Accelerating Robotic Reinforcement Learning via Parameterized Action Primitives](https://mlsanity.com/paper/2110.15360/) (2021)
- [Reinforcement Learning with Parameterized Actions](https://mlsanity.com/paper/1509.01644/) (2015)
- [Neural Task Graphs: Generalizing to Unseen Tasks from a Single Video Demonstration](https://mlsanity.com/paper/1807.03480/) (2018)
- [Neural Dynamic Policies for End-to-End Sensorimotor Learning](https://mlsanity.com/paper/2012.02788/) (2020)

## Built on by (most cited first, 18 of 18)

- [Text2Motion: From Natural Language Instructions to Feasible Plans](https://mlsanity.com/paper/2303.12153/) (2023)
- [Grounded Decoding: Guiding Text Generation with Grounded Models for Embodied Agents](https://mlsanity.com/paper/2303.00855/) (2023)
- [Sequential Dexterity: Chaining Dexterous Policies for Long-Horizon Manipulation](https://mlsanity.com/paper/2309.00987/) (2023)
- [Long-VLA: Unleashing Long-Horizon Capability of Vision Language Action Model for Robot Manipulation](https://mlsanity.com/paper/2508.19958/) (2025)
- [KITE: Keypoint-Conditioned Policies for Semantic Manipulation](https://mlsanity.com/paper/2306.16605/) (2023)
- [Trust the PRoC3S: Solving Long-Horizon Robotics Problems with LLMs and Constraint Satisfaction](https://mlsanity.com/paper/2406.05572/) (2024)
- [Primitive Skill-based Robot Learning from Human Evaluative Feedback](https://mlsanity.com/paper/2307.15801/) (2023)
- [Open-World Task and Motion Planning via Vision-Language Model Generated Constraints](https://mlsanity.com/paper/2411.08253/) (2024)
- [What's the Move? Hybrid Imitation Learning via Salient Points](https://mlsanity.com/paper/2412.05426/) (2024)
- [Robot Task Planning Based on Large Language Model Representing Knowledge with Directed Graph Structures](https://mlsanity.com/paper/2306.05171/) (2023)
- [LIT: Large Language Model Driven Intention Tracking for Proactive Human-Robot Collaboration -- A Robot Sous-Chef Application](https://mlsanity.com/paper/2406.13787/) (2024)
- [Impedance Primitive-augmented Hierarchical Reinforcement Learning for Sequential Tasks](https://mlsanity.com/paper/2508.19607/) (2025)
- [SLAP: Shortcut Learning for Abstract Planning](https://mlsanity.com/paper/2511.01107/) (2025)
- [From Shadow to Light: Toward Safe and Efficient Policy Learning Across MPC, DeePC, RL, and LLM Agents](https://mlsanity.com/paper/2510.04076/) (2025)
- [ManiSkillFormer: Demonstration-Free Compositional Manipulation via Geometric Contracts and Agentic Skill Graph](https://mlsanity.com/paper/2609.16331/) (2026)
- [Task and Skill Planning: Hierarchical Robot Planning with Black-Box Skills](https://mlsanity.com/paper/2504.17901/) (2025)
- [NOIR 2.0: Neural Signal Operated Intelligent Robots for Everyday Activities](https://mlsanity.com/paper/2511.20848/) (2025)
- [DistAL: Distance-based Advantage Learning for VLA Fine-Tuning](https://mlsanity.com/paper/2609.18392/) (2026)

## Abstract (arXiv)

Advances in robotic skill acquisition have made it possible to build general-purpose libraries of learned skills for downstream manipulation tasks. However, naively executing these skills one after the other is unlikely to succeed without accounting for dependencies between actions prevalent in long-horizon plans. We present Sequencing Task-Agnostic Policies (STAP), a scalable framework for training manipulation skills and coordinating their geometric dependencies at planning time to solve long-horizon tasks never seen by any skill during training. Given that Q-functions encode a measure of skill feasibility, we formulate an optimization problem to maximize the joint success of all skills sequenced in a plan, which we estimate by the product of their Q-values. Our experiments indicate that this objective function approximates ground truth plan feasibility and, when used as a planning objective, reduces myopic behavior and thereby promotes long-horizon task success. We further demonstrate how STAP can be used for task and motion planning by estimating the geometric feasibility of skill sequences provided by a task planner. We evaluate our approach in simulation and on a real robot. Qualitative results and code are made available at https://sites.google.com/stanford.edu/stap.