mlsanityData to 14 Oct 2026

World Modeling Makes a Better Planner: Dual Preference Optimization for Embodied Task Planning

arXiv 2503.10480 ↗First posted 13 Mar 2025cs.CL cs.CV cs.ROcited by 26

On the map: World models → World models for navigation

also in 2 more
Abstract (arXiv)

Recent advances in large vision-language models (LVLMs) have shown promise for embodied task planning, yet they struggle with fundamental challenges like dependency constraints and efficiency. Existing approaches either solely optimize action selection or leverage world models during inference, overlooking the benefits of learning to model the world as a way to enhance planning capabilities. We propose Dual Preference Optimization (D2PO), a new learning framework that jointly optimizes state prediction and action selection through preference learning, enabling LVLMs to understand environment dynamics for better planning. To automatically collect trajectories and stepwise preference data without human annotation, we introduce a tree search mechanism for extensive exploration via trial-and-error. Extensive experiments on VoTa-Bench demonstrate that our D2PO-based method significantly outperforms existing methods and GPT-4o when applied to Qwen2-VL (7B), LLaVA-1.6 (7B), and LLaMA-3.2 (11B), achieving superior task success rates with more efficient execution paths.

Key facts. Cited by 26 papers in the map (data through 14 Oct 2026). Placed in World models for navigation, one of 11 directions in World models. Builds on 43 of its 51 references that are in the map. First posted on arXiv 13 Mar 2025.

Builds on

43 of 51 references.

References · 43

  1. 2022ReAct: Synergizing Reasoning and Acting in Language Modelscited by 8,695
  2. 2023Direct Preference Optimization: Your Language Model is Secretly a Reward Modelcited by 8,065
  3. 2025DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learningcited by 6,330
  4. 2024Qwen2-VL: Enhancing Vision-Language Model's Perception of the World at Any Resolutioncited by 4,234
  5. 2024Gemini 1.5: Unlocking multimodal understanding across millions of tokens of contextcited by 3,224
  6. 2019Dream to Control: Learning Behaviors by Latent Imaginationcited by 1,653
  7. 2023Mastering Diverse Domains through World Modelscited by 1,253
  8. 2022Code as Policies: Language Model Programs for Embodied Controlcited by 1,210
  9. 2020Mastering Atari with Discrete World Modelscited by 1,007
  10. 2017AI2-THOR: An Interactive 3D Environment for Visual AIcited by 869
  11. 2023Reasoning with Language Model is Planning with World Modelcited by 861
  12. 2019ALFRED: A Benchmark for Interpreting Grounded Instructions for Everyday Taskscited by 783
31 more
  1. 2022ProgPrompt: Generating Situated Robot Task Plans using Large Language Modelscited by 611
  2. 2022LLM-Planner: Few-Shot Grounded Planning for Embodied Agents with Large Language Modelscited by 462
  3. 2023RLHF-V: Towards Trustworthy MLLMs via Behavior Alignment from Fine-grained Correctional Human Feedbackcited by 412
  4. 2022DayDreamer: World Models for Physical Robot Learningcited by 400
  5. 2024MLLM-as-a-Judge: Assessing Multimodal LLM-as-a-Judge with Vision-Language Benchmarkcited by 372
  6. 2024Enhancing the Reasoning Ability of Multimodal Large Language Models via Mixed Preference Optimizationcited by 239
  7. 2023Building Cooperative Embodied Agents Modularly with Large Language Modelscited by 237
  8. 2023Large Language Models as Commonsense Knowledge for Large-Scale Task Planningcited by 231
  9. 2023Leveraging Pre-trained Large Language Models to Construct and Utilize World Models for Model-based Task Planningcited by 207
  10. 2024Trial and Error: Exploration-Based Trajectory Optimization for LLM Agentscited by 197
  11. 2023AdaPlanner: Adaptive Planning from Feedback with Language Modelscited by 196
  12. 2023Grounding Large Language Models in Interactive Environments with Online Reinforcement Learningcited by 195
  13. 2021Episodic Transformer for Vision-and-Language Navigationcited by 156
  14. 2024mDPO: Conditional Preference Optimization for Multimodal Large Language Modelscited by 92
  15. 2023Large Language Models as Generalizable Policies for Embodied Taskscited by 86
  16. 2023Embodied Task Planning with Large Language Modelscited by 84
  17. 2023Octopus: Embodied Vision-Language Programmer from Environmental Feedbackcited by 73
  18. 2024V-DPO: Mitigating Hallucination in Large Vision Language Models via Vision-Guided Direct Preference Optimizationcited by 67
  19. 2024LoTa-Bench: Benchmarking Language-oriented Task Planners for Embodied Agentscited by 51
  20. 2025VideoWorld: Exploring Knowledge Learning from Unlabeled Videoscited by 43
  21. 2024Generating Code World Models with Large Language Models Guided by Monte Carlo Tree Searchcited by 40
  22. 2022Prompter: Utilizing Large Language Model Prompting for a Data Efficient Embodied Instruction Followingcited by 33
  23. 2023LLM as A Robotic Brain: Unifying Egocentric Memory and Controlcited by 33
  24. 2023Vision-Language Interpreter for Robot Task Planningcited by 30
  25. 2024EPO: Hierarchical LLM Agents with Environment Preference Optimizationcited by 28
  26. 2024WALL-E: World Alignment by Rule Learning Improves World Model-based LLM Agentscited by 26
  27. 2023AlphaBlock: Embodied Finetuning for Vision-Language Reasoning in Robot Manipulationcited by 19
  28. 2023ThinkBot: Embodied Instruction Following with Thought Chain Reasoningcited by 19
  29. 2024Wonderful Team: Zero-Shot Physical Task Planning with Visual LLMscited by 11
  30. 2024SELU: Self-Learning Embodied MLLMs in Unknown Environmentscited by 6
  31. 2024Hindsight Planner: A Closed-Loop Few-Shot Planner for Embodied Instruction Followingcited by 1

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 12; 2026: 14.

26 papers.

Sort:

By area:

  1. 2025Advances and Challenges in Foundation Agents: From Brain-Inspired Intelligence to Evolutionary, Collaborative, and Safe Systemscited by 126
  2. 2025A Comprehensive Survey on World Models for Embodied AIcited by 68
  3. 2025From Word to World: Can Large Language Models be Implicit Text-based World Models?cited by 42
  4. 2025Multi-agent Embodied AI: Advances and Future Directionscited by 36
  5. 2025Planning with Reasoning using Vision Language World Modelcited by 36
  6. 2026ECHO: Terminal Agents Learn World Models for Freecited by 19
  7. 2025SEEA-R1: Tree-Structured Reinforcement Fine-Tuning for Self-Evolving Embodied Agentscited by 15
  8. 2025Reinforced Reasoning for Embodied Planningcited by 12
  9. 2025ERA: Transforming VLMs into Embodied Agents via Embodied Prior Learning and Online Reinforcement Learningcited by 10
  10. 2025World-aware Planning Narratives Enhance Large Vision-Language Model Plannercited by 9
  11. 2025Unleashing Embodied Task Planning Ability in LLMs via Reinforcement Learningcited by 9
  12. 2025BEAT: Visual Backdoor Attacks on VLM-based Embodied Agents via Contrastive Trigger Learningcited by 9
14 more
  1. 2026Imagine-then-Plan: Agent Learning from Adaptive Lookahead with World Modelscited by 8
  2. 2026RoboAgent: Chaining Basic Capabilities for Embodied Task Planningcited by 7
  3. 2025MADRA: Multi-Agent Debate for Risk-Aware Embodied Planningcited by 6
  4. 2026CoMAP: Co-Evolving World Models and Agent Policies for LLM Agentscited by 6
  5. 2026Aligning Agentic World Models via Knowledgeable Experience Learningcited by 5
  6. 2026HomeGuard: VLM-based Embodied Safeguard for Identifying Contextual Risk in Household Taskcited by 5
  7. 2026In-Context World Modeling for Robotic Controlcited by 3
  8. 2026SVLL: Staged Vision-Language Learning for Physically Grounded Embodied Task Planningcited by 1
  9. 2026Behavior-Skill: A Fine-Grained Benchmark for Evaluating Vision-Language-Action Policies in Long-Horizon Taskscited by 1
  10. 2026See Better, Foresee Better, Act Wiser: Physically Grounded Proactive Modeling and Decision Makingcited by 0
  11. 2026UniETP: Unifying Environments for Generalizable Embodied Task Planningcited by 0
  12. 2026SafeBranch: Branch-Pair Safety Alignment for Embodied Agentscited by 0
  13. 2026WMLLM: Self-Evolving Optimization Agents via Predict-Then-Act World Modelingcited by 0
  14. 2026Beyond Prediction: Steering VLM Agents with Retrospective World Modelingcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2026World-Value-Action Model: Implicit Planning for Vision-Language-Action Systems
  2. 2026Prioritized Rollouts for Efficient World Model-based Vision-Language-Action Policy Optimization
  3. 2025WorldPrediction: A Benchmark for High-level World Modeling and Long-horizon Procedural Planning
  4. 2026Plan-Bench: Can Existing VLMs Go Well Beyond Perception to Reason and Plan in Embodied Worlds?
  5. 2026ACID: Action Consistency via Inverse Dynamics for Planning with World Models
  6. 2026The Planning Limits of Latent World Models
  7. 2026World Model for Robot Learning: A Comprehensive Survey
  8. 2025Structured Preference Optimization for Vision-Language Long-Horizon Task Planning