# Tactical Decision for Multi-UGV Confrontation with a Vision-Language Model-Based Commander

arXiv: https://arxiv.org/abs/2507.11079 · HTML: https://mlsanity.com/paper/2507.11079/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Placed in Embodied and 3D vision-language models, one of 16 directions in Multimodal LLMs / vision-language models.
- Builds on 16 of its 26 references that are in the map.
- First posted on arXiv 15 Jul 2025.

## Placed in

- Multimodal LLMs / vision-language models / [Embodied and 3D vision-language models](https://mlsanity.com/mllm/embodied-and-3d-vision-language-models/)

## Builds on (most cited first, 16 of 16)

- [EmbodiedGPT: Vision-Language Pre-Training via Embodied Chain of Thought](https://mlsanity.com/paper/2305.15021/) (2023)
- [Fine-Tuning Large Vision-Language Models as Decision-Making Agents via Reinforcement Learning](https://mlsanity.com/paper/2405.10292/) (2024)
- [SMART-LLM: Smart Multi-Agent Robot Task Planning using Large Language Models](https://mlsanity.com/paper/2309.10062/) (2023)
- [Learning Agile Soccer Skills for a Bipedal Robot with Deep Reinforcement Learning](https://mlsanity.com/paper/2304.13653/) (2023)
- [Self-planning Code Generation with Large Language Models](https://mlsanity.com/paper/2303.06689/) (2023)
- [Avalon's Game of Thoughts: Battle Against Deception through Recursive Contemplation](https://mlsanity.com/paper/2310.01320/) (2023)
- [Can an Embodied Agent Find Your "Cat-shaped Mug"? LLM-Guided Exploration for Zero-Shot Object Navigation](https://mlsanity.com/paper/2303.03480/) (2023)
- [Igniting Language Intelligence: The Hitchhiker's Guide From Chain-of-Thought Reasoning to Language Agents](https://mlsanity.com/paper/2311.11797/) (2023)
- [Nicer Than Humans: How do Large Language Models Behave in the Prisoner's Dilemma?](https://mlsanity.com/paper/2406.13605/) (2024)
- [UAVs Meet LLMs: Overviews and Perspectives Toward Agentic Low-Altitude Mobility](https://mlsanity.com/paper/2501.02341/) (2025)
- [VLM-Social-Nav: Socially Aware Robot Navigation through Scoring using Vision-Language Models](https://mlsanity.com/paper/2404.00210/) (2024)
- [ReplanVLM: Replanning Robotic Tasks with Visual Language Models](https://mlsanity.com/paper/2407.21762/) (2024)
- [Learning to Discuss Strategically: A Case Study on One Night Ultimate Werewolf](https://mlsanity.com/paper/2405.19946/) (2024)
- [GameVLM: A Decision-making Framework for Robotic Task Planning Based on Visual Language Models and Zero-sum Games](https://mlsanity.com/paper/2405.13751/) (2024)
- [LLCoach: Generating Robot Soccer Plans using Multi-Role Large Language Models](https://mlsanity.com/paper/2406.18285/) (2024)
- [Hierarchical Reinforcement Learning for Swarm Confrontation with High Uncertainty](https://mlsanity.com/paper/2406.07877/) (2024)

## Abstract (arXiv)

In multiple unmanned ground vehicle confrontations, autonomously evolving multi-agent tactical decisions from situational awareness remain a significant challenge. Traditional handcraft rule-based methods become vulnerable in the complicated and transient battlefield environment, and current reinforcement learning methods mainly focus on action manipulation instead of strategic decisions due to lack of interpretability. Here, we propose a vision-language model-based commander to address the issue of intelligent perception-to-decision reasoning in autonomous confrontations. Our method integrates a vision language model for scene understanding and a lightweight large language model for strategic reasoning, achieving unified perception and decision within a shared semantic space, with strong adaptability and interpretability. Unlike rule-based search and reinforcement learning methods, the combination of the two modules establishes a full-chain process, reflecting the cognitive process of human commanders. Simulation and ablation experiments validate that the proposed approach achieves a win rate of over 80% compared with baseline models.