# Training-free Generation of Temporally Consistent Rewards from VLMs

arXiv: https://arxiv.org/abs/2507.04789 · HTML: https://mlsanity.com/paper/2507.04789/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Placed in Embodied and 3D vision-language models, one of 16 directions in Multimodal LLMs / vision-language models.
- First posted on arXiv 7 Jul 2025.

## Placed in

- Multimodal LLMs / vision-language models / [Embodied and 3D vision-language models](https://mlsanity.com/mllm/embodied-and-3d-vision-language-models/)

## Built on by (most cited first, 3 of 3)

- [VisualTrans: A Benchmark for Real-World Visual Transformation Reasoning](https://mlsanity.com/paper/2508.04043/) (2025)
- [Automating Potential-based Reward Shaping with Vision Language Model Guidance](https://mlsanity.com/paper/2606.27180/) (2026)
- [PonderPounce: A Pretrained MLLM as an Episode Context Engine for Robot Control](https://mlsanity.com/paper/2608.24115/) (2026)

## Abstract (arXiv)

Recent advances in vision-language models (VLMs) have significantly improved performance in embodied tasks such as goal decomposition and visual comprehension. However, providing accurate rewards for robotic manipulation without fine-tuning VLMs remains challenging due to the absence of domain-specific robotic knowledge in pre-trained datasets and high computational costs that hinder real-time applicability. To address this, we propose T2-VLM, a novel training-free, temporally consistent framework that generates accurate rewards through tracking the status changes in VLM-derived subgoals. Specifically, our method first queries the VLM to establish spatially aware subgoals and an initial completion estimate before each round of interaction. We then employ a Bayesian tracking algorithm to update the goal completion status dynamically, using subgoal hidden states to generate structured rewards for reinforcement learning (RL) agents. This approach enhances long-horizon decision-making and improves failure recovery capabilities with RL. Extensive experiments indicate that T2-VLM achieves state-of-the-art performance in two robot manipulation benchmarks, demonstrating superior reward accuracy with reduced computation consumption. We believe our approach not only advances reward generation techniques but also contributes to the broader field of embodied AI. Project website: https://t2-vlm.github.io/.