# Video Captioning in Low-Light Conditions through Efficient Uncertainty-Aware Caption Correction

arXiv: https://arxiv.org/abs/2609.31697 · HTML: https://mlsanity.com/paper/2609.31697/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Placed in Prompt learning and adaptation of CLIP, one of 16 directions in Multimodal LLMs / vision-language models.
- Builds on 8 of its 13 references that are in the map.
- First posted on arXiv 18 Sep 2026.

## Placed in

- Multimodal LLMs / vision-language models / [Prompt learning and adaptation of CLIP](https://mlsanity.com/mllm/prompt-learning-and-adaptation-of-clip/)

## Builds on (most cited first, 8 of 8)

- [Video-LLaVA: Learning United Visual Representation by Alignment Before Projection](https://mlsanity.com/paper/2311.10122/) (2023)
- [MVBench: A Comprehensive Multi-modal Video Understanding Benchmark](https://mlsanity.com/paper/2311.17005/) (2023)
- [Video-ChatGPT: Towards Detailed Video Understanding via Large Vision and Language Models](https://mlsanity.com/paper/2306.05424/) (2023)
- [LLaMA-VID: An Image is Worth 2 Tokens in Large Language Models](https://mlsanity.com/paper/2311.17043/) (2023)
- [Video Understanding with Large Language Models: A Survey](https://mlsanity.com/paper/2312.17432/) (2023)
- [PG-Video-LLaVA: Pixel Grounding Large Video-Language Models](https://mlsanity.com/paper/2311.13435/) (2023)
- [ARID: A New Dataset for Recognizing Action in the Dark](https://mlsanity.com/paper/2006.03876/) (2020)
- [Going Deeper into Recognizing Actions in Dark Environments: A Comprehensive Benchmark Study](https://mlsanity.com/paper/2202.09545/) (2022)

## Abstract (arXiv)

Low-light conditions can significantly degrade the ability of vision-language models (VLMs) to accurately describe human actions in videos. In this work, I propose an efficient uncertainty-aware representation correction framework for improving captions generated by VideoChat2 under real-world low-light conditions. Instead of fine-tuning the underlying VLM, the proposed framework introduces a lightweight sparse Gaussian process-based error estimation module between the projection layer and the language model to correct the intermediate representation. The correction module learns to estimate the residual between the original projected representation and a verified target representation, which is then adaptively scaled using a newly formulated uncertainty-aware coefficient and added to the original representation. To further improve residual estimation, I introduce a partitioned combined-kernel design. The correction model is trained separately using only 44 samples from the ARID dataset and requires only a small additional computational overhead during inference. The effectiveness of the proposed correction is evaluated through quantitative residual prediction and qualitative analysis of the generated captions. Although VideoChat2 is used in my experiments, the proposed framework is designed to be applicable to other compatible VLM architectures. Code Availability: The implementation accompanying this work is publicly available at:hrefhttps://github.com/areferezaee/Low-rank-SVGP-NP-updatehttps://github.com/areferezaee/Low-rank-SVGP-NP-update