# STER-VLM: Spatio-Temporal With Enhanced Reference Vision-Language Models

arXiv: https://arxiv.org/abs/2508.13470 · HTML: https://mlsanity.com/paper/2508.13470/

## Key facts

- Cited by 9 papers in the map (data through 14 Oct 2026).
- Placed in Driving VLMs and open-vocabulary detection, one of 16 directions in Multimodal LLMs / vision-language models.
- Builds on 22 of its 32 references that are in the map.
- First posted on arXiv 19 Aug 2025.

## Placed in

- Multimodal LLMs / vision-language models / [Driving VLMs and open-vocabulary detection](https://mlsanity.com/mllm/driving-vlms-and-open-vocabulary-detection/)

## Builds on (most cited first, 22 of 22)

- [LoRA: Low-Rank Adaptation of Large Language Models](https://mlsanity.com/paper/2106.09685/) (2021)
- [Improved Baselines with Visual Instruction Tuning](https://mlsanity.com/paper/2310.03744/) (2023)
- [InternVL3: Exploring Advanced Training and Test-Time Recipes for Open-Source Multimodal Models](https://mlsanity.com/paper/2504.10479/) (2025)
- [BDD100K: A Diverse Driving Dataset for Heterogeneous Multitask Learning](https://mlsanity.com/paper/1805.04687/) (2018)
- [CogVLM: Visual Expert for Pretrained Language Models](https://mlsanity.com/paper/2311.03079/) (2023)
- [Better Zero-Shot Reasoning with Role-Play Prompting](https://mlsanity.com/paper/2308.07702/) (2023)
- [What does CLIP know about a red circle? Visual prompt engineering for VLMs](https://mlsanity.com/paper/2304.06712/) (2023)
- [CPT: Colorful Prompt Tuning for Pre-trained Vision-Language Models](https://mlsanity.com/paper/2109.11797/) (2021)
- [SUTD-TrafficQA: A Question Answering Benchmark and an Efficient Network for Video Reasoning over Traffic Events](https://mlsanity.com/paper/2103.15538/) (2021)
- [LLaVA-ST: A Multimodal Large Language Model for Fine-Grained Spatial-Temporal Understanding](https://mlsanity.com/paper/2501.08282/) (2025)
- [CityLLaVA: Efficient Fine-Tuning for VLMs in City Scenario](https://mlsanity.com/paper/2405.03194/) (2024)
- [TUMTraffic-VideoQA: A Benchmark for Unified Spatio-Temporal Video Understanding in Traffic Scenes](https://mlsanity.com/paper/2502.02449/) (2025)
- [When language and vision meet road safety: leveraging multimodal large language models for video-based traffic accident analysis](https://mlsanity.com/paper/2501.10604/) (2025)
- [SurveillanceVQA-589K: A Benchmark for Comprehensive Surveillance Video-Language Understanding with Large Models](https://mlsanity.com/paper/2505.12589/) (2025)
- [WTS: A Pedestrian-Centric Traffic Video Dataset for Fine-grained Spatial-Temporal Understanding](https://mlsanity.com/paper/2407.15350/) (2024)
- [The 8th AI City Challenge](https://mlsanity.com/paper/2404.09432/) (2024)
- [Improving Generalization in Visual Reasoning via Self-Ensemble](https://mlsanity.com/paper/2410.20883/) (2024)
- [RoadSocial: A Diverse VideoQA Dataset and Benchmark for Road Event Understanding from Social Video Narratives](https://mlsanity.com/paper/2503.21459/) (2025)
- [A Study of Situational Reasoning for Traffic Understanding](https://mlsanity.com/paper/2306.02520/) (2023)
- [A Lightweight Moment Retrieval System with Global Re-Ranking and Robust Adaptive Bidirectional Temporal Search](https://mlsanity.com/paper/2504.09298/) (2025)
- [CrashSage: A Large Language Model-Centered Framework for Contextual and Interpretable Traffic Crash Analysis](https://mlsanity.com/paper/2505.07853/) (2025)
- [Towards Efficient and Robust Moment Retrieval System: A Unified Framework for Multi-Granularity Models and Temporal Reranking](https://mlsanity.com/paper/2504.08384/) (2025)

## Built on by (most cited first, 9 of 9)

- [ITSELF: Attention Guided Fine-Grained Alignment for Vision-Language Retrieval](https://mlsanity.com/paper/2601.01024/) (2026)
- [ViKey: Enhancing Temporal Understanding in Videos via Visual Prompting](https://mlsanity.com/paper/2603.23186/) (2026)
- [ESC: Emotional Self-Correction for Reliable Vision-Language Models](https://mlsanity.com/paper/2607.02089/) (2026)
- [UniTraffic-Agent: Unified Traffic Video Reasoning for AI City Challenge 2026 Track 3 with Two Out-of-Domain Evaluations](https://mlsanity.com/paper/2608.13031/) (2026)
- [MADTempo: An Interactive System for Multi-Event Temporal Video Retrieval with Query Augmentation](https://mlsanity.com/paper/2512.12929/) (2025)
- [Efficient INT8 Single-Image Super-Resolution via Deployment-Aware Quantization and Teacher-Guided Training](https://mlsanity.com/paper/2604.20291/) (2026)
- [Bridging the Training-Deployment Gap: Gated Encoding and Multi-Scale Refinement for Efficient Quantization-Aware Image Enhancement](https://mlsanity.com/paper/2604.21743/) (2026)
- [SafeLens: Deliberate and Efficient Video Guardrails with Fast-and-Slow Screening](https://mlsanity.com/paper/2605.17610/) (2026)
- [Zero-Shot Traffic Accident Detection via a Coarse-to-Fine VLM-Tracking Pipeline](https://mlsanity.com/paper/2608.08867/) (2026)

## Abstract (arXiv)

Vision-language models (VLMs) have emerged as powerful tools for enabling automated traffic analysis; however, current approaches often demand substantial computational resources and struggle with fine-grained spatio-temporal understanding. This paper introduces STER-VLM, a computationally efficient framework that enhances VLM performance through (1) caption decomposition to tackle spatial and temporal information separately, (2) temporal frame selection with best-view filtering for sufficient temporal information, and (3) reference-driven understanding for capturing fine-grained motion and dynamic context and (4) curated visual/textual prompt techniques. Experimental results on the WTS citekong2024wts and BDD citeBDD datasets demonstrate substantial gains in semantic richness and traffic scene interpretation. Our framework is validated through a decent test score of 55.655 in the AI City Challenge 2025 Track 2, showing its effectiveness in advancing resource-efficient and accurate traffic analysis for real-world applications.