Tactical Decision for Multi-UGV Confrontation with a Vision-Language Model-Based Commander
On the map: Multimodal LLMs / vision-language models → Embodied and 3D vision-language models
Abstract (arXiv)
In multiple unmanned ground vehicle confrontations, autonomously evolving multi-agent tactical decisions from situational awareness remain a significant challenge. Traditional handcraft rule-based methods become vulnerable in the complicated and transient battlefield environment, and current reinforcement learning methods mainly focus on action manipulation instead of strategic decisions due to lack of interpretability. Here, we propose a vision-language model-based commander to address the issue of intelligent perception-to-decision reasoning in autonomous confrontations. Our method integrates a vision language model for scene understanding and a lightweight large language model for strategic reasoning, achieving unified perception and decision within a shared semantic space, with strong adaptability and interpretability. Unlike rule-based search and reinforcement learning methods, the combination of the two modules establishes a full-chain process, reflecting the cognitive process of human commanders. Simulation and ablation experiments validate that the proposed approach achieves a win rate of over 80% compared with baseline models.
Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in Embodied and 3D vision-language models, one of 16 directions in Multimodal LLMs / vision-language models. Builds on 16 of its 26 references that are in the map. First posted on arXiv 15 Jul 2025.
Builds on
16 of 26 references.
Closest ancestors · 5
Same direction, newest first
- 2024ReplanVLM: Replanning Robotic Tasks with Visual Language Models31 Jul
- 2024GameVLM: A Decision-making Framework for Robotic Task Planning Based on Visual Language Models and Zero-sum Games22 May
- 2024Fine-Tuning Large Vision-Language Models as Decision-Making Agents via Reinforcement Learning16 May
- 2024VLM-Social-Nav: Socially Aware Robot Navigation through Scoring using Vision-Language Models30 Mar
- 2023EmbodiedGPT: Vision-Language Pre-Training via Embodied Chain of Thought24 May
Other references · 11
- 2023SMART-LLM: Smart Multi-Agent Robot Task Planning using Large Language Modelscited by 153
- 2023Learning Agile Soccer Skills for a Bipedal Robot with Deep Reinforcement Learningcited by 133
- 2023Self-planning Code Generation with Large Language Modelscited by 124
- 2023Avalon's Game of Thoughts: Battle Against Deception through Recursive Contemplationcited by 104
- 2023Can an Embodied Agent Find Your "Cat-shaped Mug"? LLM-Guided Exploration for Zero-Shot Object Navigationcited by 74
- 2023Igniting Language Intelligence: The Hitchhiker's Guide From Chain-of-Thought Reasoning to Language Agentscited by 72
- 2024Nicer Than Humans: How do Large Language Models Behave in the Prisoner's Dilemma?cited by 63
- 2025UAVs Meet LLMs: Overviews and Perspectives Toward Agentic Low-Altitude Mobilitycited by 49
- 2024Learning to Discuss Strategically: A Case Study on One Night Ultimate Werewolfcited by 24
- 2024LLCoach: Generating Robot Soccer Plans using Multi-Role Large Language Modelscited by 3
- 2024Hierarchical Reinforcement Learning for Swarm Confrontation with High Uncertaintycited by 2
Built on by
Not cited yet.
Similar, not cited
Close in wording, no citation either way
- 2026DoublesEval: Diagnosing Multi-Agent Tactical Reasoning in Vision-Language Models via Professional Doubles Badminton
- 2026VLGOR: Visual-Language Knowledge Guided Offline Reinforcement Learning for Generalizable Agents
- 2026Vision-and-Language Navigation for UAVs: Progress, Challenges, and a Research Roadmap
- 2025VS-Bench: Evaluating VLMs for Strategic Abilities in Multi-Agent Environments
- 2024Data-Driven Distributed Common Operational Picture from Heterogeneous Platforms using Multi-Agent Reinforcement Learning
- 2025VAGEN: Reinforcing World Model Reasoning for Multi-Turn VLM Agents
- 2025UAV-VLN: End-to-End Vision Language guided Navigation for UAVs
- 2025G1: Bootstrapping Perception and Reasoning Abilities of Vision-Language Model via Reinforcement Learning