AVC-DPO: Aligned Video Captioning via Direct Preference Optimization
On the map: Multimodal LLMs / vision-language models → MLLM training and visual instruction tuning
also in 1 more
Abstract (arXiv)
Although video multimodal large language models (video MLLMs) have achieved substantial progress in video captioning tasks, it remains challenging to adjust the focal emphasis of video captions according to human preferences. To address this limitation, we propose Aligned Video Captioning via Direct Preference Optimization (AVC-DPO), a post-training framework designed to enhance captioning capabilities in video MLLMs through preference alignment. Our approach designs enhanced prompts that specifically target temporal dynamics and spatial information-two key factors that humans care about when watching a video-thereby incorporating human-centric preferences. AVC-DPO leverages the same foundation model's caption generation responses under varied prompt conditions to conduct preference-aware training and caption alignment. Using this framework, we have achieved exceptional performance in the LOVE@CVPR'25 Workshop Track 1A: Video Detailed Captioning Challenge, achieving first place on the Video Detailed Captioning (VDC) benchmark according to the VDCSCORE evaluation metric.
Key facts. Cited by 2 papers in the map (data through 14 Oct 2026). Placed in MLLM training and visual instruction tuning, one of 16 directions in Multimodal LLMs / vision-language models. Builds on 19 of its 20 references that are in the map. First posted on arXiv 2 Jul 2025.
Builds on
19 of 20 references.
Closest ancestors · 2
Same direction, newest first
- 2024LLaVA-Video: Video Instruction Tuning With Synthetic Data3 Oct
- 2024Qwen2-VL: Enhancing Vision-Language Model's Perception of the World at Any Resolution18 Sep
Other references · 17
- 2023Direct Preference Optimization: Your Language Model is Secretly a Reward Modelcited by 8,065
- 2025Qwen2.5-VL Technical Reportcited by 5,587
- 2024LlamaFactory: Unified Efficient Fine-Tuning of 100+ Language Modelscited by 1,464
- 2025Inference-Time Scaling for Generalist Reward Modelingcited by 239
- 2024VideoChat-Flash: Hierarchical Compression for Long-Context Video Modelingcited by 220
- 2024AuroraCap: Efficient, Performant Video Detailed Captioning and a New Benchmarkcited by 140
- 2024Aria: An Open Multimodal Native Mixture-of-Experts Modelcited by 138
- 2025Describe Anything: Detailed Localized Image and Video Captioningcited by 84
- 2025Chain-of-Frames: Advancing Video Understanding in Multimodal LLMs via Frame-Aware Reasoningcited by 37
- 2025VideoCap-R1: Enhancing MLLMs for Video Captioning via Structured Thinkingcited by 30
- 2025VidBridge-R1: Bridging QA and Captioning for RL-based Video Understanding Models with Intermediate Proxy Taskscited by 14
- 2024Progress-Aware Video Frame Captioningcited by 11
5 more
- 2025Caption Anything in Video: Fine-grained Object-centric Captioning via Spatiotemporal Multimodal Promptingcited by 10
- 2025VCapsBench: A Large-scale Fine-grained Benchmark for Video Caption Quality Evaluationcited by 9
- 2025ActionArt: Advancing Multimodal Large Models for Fine-Grained Human-Centric Video Understandingcited by 7
- 2024Personalized Video Summarization by Multimodal Video Understandingcited by 3
- 2025Cockatiel: Ensembling Synthetic and Human Preferenced Training for Detailed Video Captioncited by 2
Built on by
Citing papers per year: 2025: 1; 2026: 1.
2 papers.
Similar, not cited
Close in wording, no citation either way
- 2025SynPO: Synergizing Descriptiveness and Preference Optimization for Video Detailed Captioning
- 2026Learning to Rank Caption Chains for Video-Text Alignment
- 2025Captioning for Text-Video Retrieval via Dual-Group Direct Preference Optimization
- 2024Enhancing Multimodal LLM for Detailed and Accurate Video Captioning using Multi-Round Preference Optimization
- 2025VistaDPO: Video Hierarchical Spatial-Temporal Direct Preference Optimization for Large Video Models
- 2026AVCap: Reinforcing Audio-Video Joint Caption with Detail-Aware Reward
- 2026Beyond Binary Preferences: Graded Preference Optimization for Limb-Motion Captioning
- 2025Self-alignment of Large Video Language Models with Refined Regularized Preference Optimization