# ViCo: Engaging Video Comment Generation with Human Preference Rewards arXiv: https://arxiv.org/abs/2308.11171 ยท HTML: https://mlsanity.com/paper/2308.11171/ ## Key facts - Cited by 8 papers in the map (data through 14 Oct 2026). - Placed in Diffusion and multimodal model alignment, one of 10 directions in LLM alignment / RLHF / preference optimisation. - Builds on 28 of its 46 references that are in the map. - First posted on arXiv 22 Aug 2023. ## Placed in - LLM alignment / RLHF / preference optimisation / [Diffusion and multimodal model alignment](https://mlsanity.com/alignment/diffusion-and-multimodal-model-alignment/) ## Builds on (most cited first, 28 of 28) - [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015) - [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018) - [Learning Transferable Visual Models From Natural Language Supervision](https://mlsanity.com/paper/2103.00020/) (2021) - [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020) - [GPT-4 Technical Report](https://mlsanity.com/paper/2303.08774/) (2023) - [Training language models to follow instructions with human feedback](https://mlsanity.com/paper/2203.02155/) (2022) - [Proximal Policy Optimization Algorithms](https://mlsanity.com/paper/1707.06347/) (2017) - [BLIP-2: Bootstrapping Language-Image Pre-training with Frozen Image Encoders and Large Language Models](https://mlsanity.com/paper/2301.12597/) (2023) - [Flamingo: a Visual Language Model for Few-Shot Learning](https://mlsanity.com/paper/2204.14198/) (2022) - [CIDEr: Consensus-based Image Description Evaluation](https://mlsanity.com/paper/1411.5726/) (2014) - [Dense-Captioning Events in Videos](https://mlsanity.com/paper/1705.00754/) (2017) - [Localizing Moments in Video with Natural Language](https://mlsanity.com/paper/1708.01641/) (2017) - [Multimodal Few-Shot Learning with Frozen Language Models](https://mlsanity.com/paper/2106.13884/) (2021) - [Towards Automatic Learning of Procedures from Web Instructional Videos](https://mlsanity.com/paper/1703.09788/) (2017) - [Sequence Level Training with Recurrent Neural Networks](https://mlsanity.com/paper/1511.06732/) (2015) - [ClipCap: CLIP Prefix for Image Captioning](https://mlsanity.com/paper/2111.09734/) (2021) - [Self-critical Sequence Training for Image Captioning](https://mlsanity.com/paper/1612.00563/) (2016) - [GIT: A Generative Image-to-text Transformer for Vision and Language](https://mlsanity.com/paper/2205.14100/) (2022) - [Visual Storytelling](https://mlsanity.com/paper/1604.03968/) (2016) - [SwinBERT: End-to-End Transformers with Sparse Attention for Video Captioning](https://mlsanity.com/paper/2111.13196/) (2021) - [End-to-end Generative Pretraining for Multimodal Video Captioning](https://mlsanity.com/paper/2201.08264/) (2022) - [CLIP4Caption: CLIP for Video Caption](https://mlsanity.com/paper/2110.06615/) (2021) - [Memory-Attended Recurrent Network for Video Captioning](https://mlsanity.com/paper/1905.03966/) (2019) - [Engaging Image Captioning Via Personality](https://mlsanity.com/paper/1810.10665/) (2018) - [Describing like humans: on diversity in image captioning](https://mlsanity.com/paper/1903.12020/) (2019) - [Video Storytelling: Textual Summaries for Events](https://mlsanity.com/paper/1807.09418/) (2018) - [LiveBot: Generating Live Video Comments Based on Visual and Textual Contexts](https://mlsanity.com/paper/1809.04938/) (2018) - [VTC: Improving Video-Text Retrieval with User Comments](https://mlsanity.com/paper/2210.10820/) (2022) ## Built on by (most cited first, 8 of 8) - [HOTVCOM: Generating Buzzworthy Comments for Videos](https://mlsanity.com/paper/2409.15196/) (2024) - [HotComment: A Benchmark for Evaluating Popularity of Online Comments](https://mlsanity.com/paper/2604.25614/) (2026) - [Semantic Frame Aggregation-based Transformer for Live Video Comment Generation](https://mlsanity.com/paper/2510.26978/) (2025) - [Laugh, Relate, Engage: Stylized Comment Generation for Short Videos](https://mlsanity.com/paper/2511.03757/) (2025) - [ComVi: Context-Aware Optimized Comment Display in Video Playback](https://mlsanity.com/paper/2603.26173/) (2026) - [LOLGORITHM: Funny Comment Generation Agent For Short Videos](https://mlsanity.com/paper/2604.09729/) (2026) - [Let the Bullets Fly: Multimodal Fake News Detection with Temporal-Aligned Generative Danmaku](https://mlsanity.com/paper/2608.22832/) (2026) - [Text-to-seed generation: Training-free open-vocabulary seeded semantic segmentation via re-purposing diffusion as text-guided seed generator](https://mlsanity.com/paper/2608.26624/) (2026) ## Abstract (arXiv) Engaging video comments play an important role in video social media, as they are the carrier of feelings, thoughts, or humor of the audience. Preliminary works have made initial exploration for video comment generation by adopting caption-style encoder-decoder models. However, comment generation presents some unique challenges distinct from caption generation, which makes these methods somewhat less effective at generating engaging comments. In contrast to the objective and descriptive nature of captions, comments tend to be inherently subjective, making it hard to quantify and evaluate the engagement of comments. Furthermore, the scarcity of truly engaging comments brings difficulty to collecting enough high-quality training examples. In this paper, we propose ViCo with three novel designs to tackle the above challenges for generating engaging Video Comments. Firstly, to quantify the engagement of comments, we utilize the number of "likes" each comment receives as a proxy of human preference after an appropriate debiasing procedure. Secondly, to automatically evaluate the engagement of comments, we train a reward model to align its judgment to the above proxy. Our user studies indicate that this reward model effectively aligns with human judgments. Lastly, to alleviate the scarcity of high-quality comments, an initial generator is trained on readily available but noisy data to generate comments. Then the reward model is employed to offer feedback on the generated comments, thus optimizing the initial generator. To facilitate the research of video commenting, we collect a large video comment-dataset (ViCo-20k) with rich metadata from a popular video website. Experiments on ViCo-20k show that the comments generated by our ViCo model exhibit the best performance in terms of both quantitative and qualitative results, particularly when engagement is considered.