PolySmart @ TRECVid 2024 Video Captioning (VTT)
On the map: Multimodal LLMs / vision-language models → Video understanding with MLLMs
Abstract (arXiv)
In this paper, we present our methods and results for the Video-To-Text (VTT) task at TRECVid 2024, exploring the capabilities of Vision-Language Models (VLMs) like LLaVA and LLaVA-NeXT-Video in generating natural language descriptions for video content. We investigate the impact of fine-tuning VLMs on VTT datasets to enhance description accuracy, contextual relevance, and linguistic consistency. Our analysis reveals that fine-tuning substantially improves the model's ability to produce more detailed and domain-aligned text, bridging the gap between generic VLM tasks and the specialized needs of VTT. Experimental results demonstrate that our fine-tuned model outperforms baseline VLMs across various evaluation metrics, underscoring the importance of domain-specific tuning for complex VTT tasks.
Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in Video understanding with MLLMs, one of 16 directions in Multimodal LLMs / vision-language models. First posted on arXiv 20 Dec 2024.
Builds on
0 of 0 references.
No references here yet.
Built on by
Not cited yet.
Similar, not cited
Close in wording, no citation either way
- 2024Distilling Vision-Language Models on Millions of Videos
- 2026Advancing Video-Text Pretraining with Multi-View Captions
- 2024MiniGPT4-Video: Advancing Multimodal LLMs for Video Understanding with Interleaved Visual-Textual Tokens
- 2025Q-Adapter: Visual Query Adapter for Extracting Textually-related Features in Video Captioning
- 2024Vript: A Video Is Worth Thousands of Words
- 2025Trust the Model: Compact VLMs as In-Context Judges for Image-Text Data Quality
- 2025VTD-CLIP: Video-to-Text Discretization via Prompting CLIP
- 2023VLM-Eval: A General Evaluation on Video Large Language Models