mlsanityData to 7 Oct 2026

Chain-of-Thought Predictive Control

arXiv 2304.00776 ↗First posted 3 Apr 2023cs.LG cs.AI cs.ROcited by 18

On the map: LLM reasoning → Spatial and embodied reasoning

also in 1 more
Abstract (arXiv)

We study generalizable policy learning from demonstrations for complex low-level control (e.g., contact-rich object manipulations). We propose a novel hierarchical imitation learning method that utilizes sub-optimal demos. Firstly, we propose an observation space-agnostic approach that efficiently discovers the multi-step subskill decomposition of the demos in an unsupervised manner. By grouping temporarily close and functionally similar actions into subskill-level demo segments, the observations at the segment boundaries constitute a chain of planning steps for the task, which we refer to as the chain-of-thought (CoT). Next, we propose a Transformer-based design that effectively learns to predict the CoT as the subskill-level guidance. We couple action and subskill predictions via learnable prompt tokens and a hybrid masking strategy, which enable dynamically updated guidance at test time and improve feature representation of the trajectory for generalizable policy learning. Our method, Chain-of-Thought Predictive Control (CoTPC), consistently surpasses existing strong baselines on challenging manipulation tasks with sub-optimal demos.

Key facts. Cited by 18 papers in the map (data through 7 Oct 2026). Placed in Spatial and embodied reasoning, one of 17 directions in LLM reasoning. Builds on 83 of its 108 references that are in the map. First posted on arXiv 3 Apr 2023.

Builds on

83 of 108 references.

Closest ancestors · 1

Same direction, newest first

  1. 2022Chain of Thought Imitation with Procedure Cloning22 May

Other references · 82

  1. 2017Attention Is All You Needcited by 59,932
  2. 2020Language Models are Few-Shot Learnerscited by 32,199
  3. 2020Denoising Diffusion Probabilistic Modelscited by 20,851
  4. 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,302
  5. 2021Training Verifiers to Solve Math Word Problemscited by 9,743
  6. 2020End-to-End Object Detection with Transformerscited by 7,384
  7. 2021Masked Autoencoders Are Scalable Vision Learnerscited by 7,339
  8. 2016PointNet: Deep Learning on Point Sets for 3D Classification and Segmentationcited by 5,223
  9. 2020Exploring Simple Siamese Representation Learningcited by 2,853
  10. 2023Diffusion Policy: Visuomotor Policy Learning via Action Diffusioncited by 2,730
  11. 2014Empirical Evaluation of Gated Recurrent Neural Networks on Sequence Modelingcited by 2,341
  12. 2022Do As I Can, Not As I Say: Grounding Language in Robotic Affordancescited by 2,302
38 more
  1. 2021Learning to Prompt for Vision-Language Modelscited by 2,269
  2. 2023PaLM-E: An Embodied Multimodal Language Modelcited by 2,037
  3. 2022RT-1: Robotics Transformer for Real-World Control at Scalecited by 2,007
  4. 2020Offline Reinforcement Learning: Tutorial, Review, and Perspectives on Open Problemscited by 1,873
  5. 2020Conservative Q-Learning for Offline Reinforcement Learningcited by 1,777
  6. 2016Generative Adversarial Imitation Learningcited by 1,576
  7. 2021Decision Transformer: Reinforcement Learning via Sequence Modelingcited by 1,528
  8. 2020D4RL: Datasets for Deep Data-Driven Reinforcement Learningcited by 1,406
  9. 2022Planning with Diffusion for Flexible Behavior Synthesiscited by 1,023
  10. 2018DeepMind Control Suitecited by 935
  11. 2021Show Your Work: Scratchpads for Intermediate Computation with Language Modelscited by 871
  12. 2019Stabilizing Off-Policy Q-Learning via Bootstrapping Error Reductioncited by 833
  13. 2019ALFRED: A Benchmark for Interpreting Grounded Instructions for Everyday Taskscited by 773
  14. 2018Diversity is All You Need: Learning Skills without a Reward Functioncited by 734
  15. 2021What Matters in Learning from Offline Human Demonstrations for Robot Manipulationcited by 699
  16. 2017Learning Complex Dexterous Manipulation with Deep Reinforcement Learning and Demonstrationscited by 685
  17. 2020AWAC: Accelerating Online Reinforcement Learning with Offline Datasetscited by 647
  18. 2017Program Induction by Rationale Generation : Learning to Solve and Explain Algebraic Word Problemscited by 642
  19. 2021Offline Reinforcement Learning as One Big Sequence Modeling Problemcited by 636
  20. 2020SAPIEN: A SimulAted Part-based Interactive ENvironmentcited by 584
  21. 2022Perceiver-Actor: A Multi-Task Transformer for Robotic Manipulationcited by 545
  22. 2022Diffusion Policies as an Expressive Policy Class for Offline Reinforcement Learningcited by 543
  23. 2022Is Conditional Generative Modeling all you need for Decision-Making?cited by 504
  24. 2022MineDojo: Building Open-Ended Embodied Agents with Internet-Scale Knowledgecited by 469
  25. 2021Implicit Behavioral Cloningcited by 442
  26. 2018PVNet: Pixel-wise Voting Network for 6DoF Pose Estimationcited by 385
  27. 2019Relay Policy Learning: Solving Long-Horizon Tasks via Imitation and Reinforcement Learningcited by 380
  28. 2019Explain Yourself! Leveraging Language Models for Commonsense Reasoningcited by 367
  29. 2020Transporter Networks: Rearranging the Visual World for Robotic Manipulationcited by 342
  30. 2023ManiSkill2: A Unified Benchmark for Generalizable Manipulation Skillscited by 313
  31. 2019Learning Latent Plans from Playcited by 310
  32. 2022Behavior Transformers: Cloning k modes with one stonecited by 299
  33. 2017Deep Imitation Learning for Complex Manipulation Tasks from Virtual Reality Teleoperationcited by 286
  34. 2020Transformers as Soft Reasoners over Languagecited by 265
  35. 2021Offline Reinforcement Learning with Fisher Divergence Critic Regularizationcited by 237
  36. 2020Accelerating Reinforcement Learning with Learned Skill Priorscited by 220
  37. 2021ManiSkill: Generalizable Manipulation Skill Benchmark with Large-Scale Demonstrationscited by 193
  38. 2017DART: Noise Injection for Robust Imitation Learningcited by 187

50 most cited of 82 All 82 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 5; 2024: 9; 2025: 3; 2026: 1.

18 papers.

Sort:

By area:

  1. 2023Towards Better Chain-of-Thought Prompting Strategies: A Surveycited by 62
  2. 2024Point Cloud Matters: Rethinking the Impact of Different Observation Spaces on Robot Learningcited by 45
  3. 2024Diffusion World Model: Future Modeling Beyond Step-by-Step Rollout for Offline Reinforcement Learningcited by 31
  4. 2023Kinematic-aware Prompting for Generalizable Articulated Object Manipulation with LLMscited by 19
  5. 2024Play to the Score: Stage-Guided Dynamic Multi-Sensory Fusion for Robotic Manipulationcited by 19
  6. 2024SAM-E: Leveraging Visual Foundation Model with Sequence Imitation for Embodied Manipulationcited by 18
  7. 2023EasyHeC: Accurate and Automatic Hand-eye Calibration via Differentiable Rendering and Space Explorationcited by 16
  8. 2023CoT3DRef: Chain-of-Thoughts Data-Efficient 3D Visual Groundingcited by 13
  9. 2025Think Small, Act Big: Primitive Prompt Learning for Lifelong Robot Manipulationcited by 13
  10. 2023Efficient Planning with Latent Diffusioncited by 11
  11. 2025Rethinking Latent Redundancy in Behavior Cloning: An Information Bottleneck Approach for Robot Manipulationcited by 10
  12. 2024MaxMI: A Maximal Mutual Information Criterion for Manipulation Concept Discoverycited by 5
6 more
  1. 2024InfoCon: Concept Discovery with Generative and Discriminative Informativenesscited by 3
  2. 2024Multi-State-Action Tokenisation in Decision Transformers for Multi-Discrete Action Spacescited by 0
  3. 2024Hierarchical Diffusion Policy: manipulation trajectory generation via contact guidancecited by 0
  4. 2024M3PC: Test-time Model Predictive Control for Pretrained Masked Trajectory Modelcited by 0
  5. 2025Inverse Reinforcement Learning via Convex Optimizationcited by 0
  6. 2026Reinforcement Learning: From Algorithms To Foundation Modelscited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2022Learning Category-Level Generalizable Object Manipulation Policy via Generative Adversarial Self-Imitation Learning from Demonstrations
  2. 2025Generalizable Hierarchical Skill Learning via Object-Centric Representation
  3. 2024Learning from 10 Demos: Generalisable and Sample-Efficient Policy Learning with Oriented Affordance Frames
  4. 2025Multi-Stage Manipulation with Demonstration-Augmented Reward, Policy, and World Model Learning
  5. 2025ArticuBot: Learning Universal Articulated Object Manipulation Policy via Large Scale Simulation
  6. 2020Model-Based Inverse Reinforcement Learning from Visual Demonstrations
  7. 2026SADP: Subgoal-Aware Diffusion Policy for Long-Horizon Manipulation Learned from Foundation Model Generated Demonstrations
  8. 2025Zero-Shot Visual Generalization in Robot Manipulation