mlsanityData to 7 Oct 2026

APIVOT: Adaptive Planning with Interleaved Vision-Language Thoughts

arXiv 2607.08024 ↗First posted 9 Jul 2026cs.CV cs.AI cs.LGcited by 0

Not on the map. Why?

Abstract (arXiv)

Long-horizon robot planning requires jointly reasoning over semantic task structure and geometric feasibility. To successfully execute a task, a robot must decompose goals, select task-relevant objects, and sequence actions, while ensuring that plans satisfy spatial constraints such as limited free space and object collisions. In this work, we propose APIVOT, a VLM-based planner that adaptively interleaves language and visual thoughts for long-horizon planning. APIVOT learns to leverage language for semantic reasoning, while using visual thoughts as imagined future states for internal verification of geometric feasibility. On long-horizon kitchen tasks, APIVOT outperforms general-purpose VLMs and prior planning frameworks, achieving the largest gains in spatially constrained settings. We find that APIVOT learns meaningful modality selection behavior, demonstrating that adaptive interleaving of vision-language thoughts improves both planning success and reasoning efficiency.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 40 of its 51 references that are in the map. First posted on arXiv 9 Jul 2026.

Builds on

40 of 51 references.

References · 40

  1. 2023Lost in the Middle: How Language Models Use Long Contextscited by 3,383
  2. 2022Do As I Can, Not As I Say: Grounding Language in Robotic Affordancescited by 2,302
  3. 2022Code as Policies: Language Model Programs for Embodied Controlcited by 1,172
  4. 2022Inner Monologue: Embodied Reasoning through Planning with Language Modelscited by 1,057
  5. 2022Language Models as Zero-Shot Planners: Extracting Actionable Knowledge for Embodied Agentscited by 1,029
  6. 2025Emerging Properties in Unified Multimodal Pretrainingcited by 855
  7. 2023LLM+P: Empowering Large Language Models with Optimal Planning Proficiencycited by 496
  8. 2022LLM-Planner: Few-Shot Grounded Planning for Embodied Agents with Large Language Modelscited by 459
  9. 2025CoT-VLA: Visual Chain-of-Thought Reasoning for Vision-Language-Action Modelscited by 440
  10. 2024Robotic Control via Embodied Chain-of-Thought Reasoningcited by 340
  11. 2020Integrated Task and Motion Planningcited by 281
  12. 2025Pixel Reasoner: Incentivizing Pixel-Space Reasoning with Curiosity-Driven Reinforcement Learningcited by 267
28 more
  1. 2025Show-o2: Improved Native Unified Multimodal Modelscited by 242
  2. 2025Imagine while Reasoning in Space: Multimodal Visualization-of-Thoughtcited by 204
  3. 2025Thinking with Images for Multimodal Reasoning: Foundations, Methods, and Future Frontierscited by 193
  4. 2018PDDLStream: Integrating Symbolic Planners and Blackbox Samplers via Optimistic Adaptive Planningcited by 147
  5. 2026π0.7: a Steerable Generalist Robotic Foundation Model with Emergent Capabilitiescited by 127
  6. 2025Machine Mental Imagery: Empower Multimodal Reasoning with Latent Visual Tokenscited by 120
  7. 2025AdaCoT: Pareto-Optimal Adaptive Chain-of-Thought Triggering via Reinforcement Learningcited by 86
  8. 2023Embodied Task Planning with Large Language Modelscited by 84
  9. 2025Zebra-CoT: A Dataset for Interleaved Vision Language Reasoningcited by 78
  10. 2025Chain-of-Visual-Thought: Teaching VLMs to See and Think Better with Continuous Visual Tokenscited by 65
  11. 2024LLM3:Large Language Model-based Task and Motion Planning with Motion Failure Reasoningcited by 63
  12. 2025ThinkMorph: Emergent Properties in Multimodal Interleaved Chain-of-Thought Reasoningcited by 55
  13. 2025Training Strategies for Efficient Embodied Reasoningcited by 53
  14. 2024Guiding Long-Horizon Task and Motion Planning with Vision Language Modelscited by 46
  15. 2025Reflective Planning: Vision-Language Models for Multi-Stage Long-Horizon Robotic Manipulationcited by 40
  16. 2023CoPAL: Corrective Planning of Robot Actions with Large Language Modelscited by 39
  17. 2024RePLan: Robotic Replanning with Perception and Language Modelscited by 39
  18. 2025From Seeing to Doing: Bridging Reasoning and Decision for Robotic Manipulationcited by 34
  19. 2025TUNA: Taming Unified Visual Representations for Native Unified Multimodal Modelscited by 34
  20. 2026Beyond Language Modeling: An Exploration of Multimodal Pretrainingcited by 32
  21. 2025EO-1: An Open Unified Embodied Foundation Model for General Robot Controlcited by 30
  22. 2026BagelVLA: Enhancing Long-Horizon Manipulation via Interleaved Vision-Language-Action Generationcited by 25
  23. 2026Tuna-2: Pixel Embeddings Beat Vision Encoders for Multimodal Understanding and Generationcited by 17
  24. 2025Learning Compositional Behaviors from Demonstration and Languagecited by 15
  25. 2025Mull-Tokens: Modality-Agnostic Latent Thinkingcited by 15
  26. 2024LATTE: Learning to Think with Vision Specialistscited by 13
  27. 2024"Set It Up!": Functional Object Arrangement with Compositional Generative Modelscited by 9
  28. 2025Using VLM Reasoning to Constrain Task and Motion Planningcited by 2

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2023Look Before You Leap: Unveiling the Power of GPT-4V in Robotic Vision-Language Planning
  2. 2026Thinking in Text and Images: Interleaved Vision--Language Reasoning Traces for Long-Horizon Robot Manipulation
  3. 2025World-aware Planning Narratives Enhance Large Vision-Language Model Planner
  4. 2026Fast-ThinkAct: Efficient Vision-Language-Action Reasoning via Verbalizable Latent Planning
  5. 2025ExploreVLM: Closed-Loop Robot Exploration Task Planning with Vision-Language Models
  6. 2025Visual Planning: Let's Think Only with Images
  7. 2025Making VLMs More Robot-Friendly: Self-Critical Distillation of Low-Level Procedural Reasoning
  8. 2025ThinkAct: Vision-Language-Action Reasoning via Reinforced Visual Latent Planning