# PIGEONS: Incremental Long-Video Captioning with Persistent Identity Gallery and Evolving Overarching Narrative Summary

arXiv: https://arxiv.org/abs/2610.16066 · HTML: https://mlsanity.com/paper/2610.16066/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 51 of its 55 references that are in the map.
- First posted on arXiv 12 Oct 2026.

## Builds on (most cited first, 30 of 51)

- [Proximal Policy Optimization Algorithms](https://mlsanity.com/paper/1707.06347/) (2017)
- [DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Models](https://mlsanity.com/paper/2402.03300/) (2024)
- [Gemini 2.5: Pushing the Frontier with Advanced Reasoning, Multimodality, Long Context, and Next Generation Agentic Capabilities](https://mlsanity.com/paper/2507.06261/) (2025)
- [DAPO: An Open-Source LLM Reinforcement Learning System at Scale](https://mlsanity.com/paper/2503.14476/) (2025)
- [DINOv3](https://mlsanity.com/paper/2508.10104/) (2025)
- [Group Sequence Policy Optimization](https://mlsanity.com/paper/2507.18071/) (2025)
- [Qwen2.5-Omni Technical Report](https://mlsanity.com/paper/2503.20215/) (2025)
- [MovieChat: From Dense Token to Sparse Memory for Long Video Understanding](https://mlsanity.com/paper/2307.16449/) (2023)
- [Qwen3-Omni Technical Report](https://mlsanity.com/paper/2509.17765/) (2025)
- [ShareGPT4Video: Improving Video Understanding and Generation with Better Captions](https://mlsanity.com/paper/2406.04325/) (2024)
- [VideoChat-R1: Enhancing Spatio-Temporal Perception via Reinforcement Fine-Tuning](https://mlsanity.com/paper/2504.06958/) (2025)
- [Movie Description](https://mlsanity.com/paper/1605.03705/) (2016)
- [WorldSense: Evaluating Real-world Omnimodal Understanding for Multimodal LLMs](https://mlsanity.com/paper/2502.04326/) (2025)
- [Tarsier: Recipes for Training and Evaluating Large Video Description Models](https://mlsanity.com/paper/2407.00634/) (2024)
- [AuroraCap: Efficient, Performant Video Detailed Captioning and a New Benchmark](https://mlsanity.com/paper/2410.03051/) (2024)
- [CinePile: A Long Video Question Answering Dataset and Benchmark](https://mlsanity.com/paper/2405.08813/) (2024)
- [Seeing, Listening, Remembering, and Reasoning: A Multimodal Agent with Long-Term Memory](https://mlsanity.com/paper/2508.09736/) (2025)
- [Video ReCap: Recursive Captioning of Hour-Long Videos](https://mlsanity.com/paper/2402.13250/) (2024)
- [Tarsier2: Advancing Large Vision-Language Models from Detailed Video Description to Comprehensive Video Understanding](https://mlsanity.com/paper/2501.07888/) (2025)
- [OmniVinci: Enhancing Architecture and Data for Omni-Modal Understanding LLM](https://mlsanity.com/paper/2510.15870/) (2025)
- [LongVALE: Vision-Audio-Language-Event Benchmark Towards Time-Aware Omni-Modal Perception of Long Videos](https://mlsanity.com/paper/2411.19772/) (2024)
- [video-SALMONN 2: Caption-Enhanced Audio-Visual Large Language Models](https://mlsanity.com/paper/2506.15220/) (2025)
- [ARC-Hunyuan-Video-7B: Structured Video Comprehension of Real-World Shorts](https://mlsanity.com/paper/2507.20939/) (2025)
- [OneStory: Coherent Multi-Shot Video Generation with Adaptive Memory](https://mlsanity.com/paper/2512.07802/) (2025)
- [Mavors: Multi-granularity Video Representation for Multimodal Large Language Model](https://mlsanity.com/paper/2504.10068/) (2025)
- [VideoCap-R1: Enhancing MLLMs for Video Captioning via Structured Thinking](https://mlsanity.com/paper/2506.01725/) (2025)
- [Omni-Captioner: Data Pipeline, Models, and Benchmark for Omni Detailed Perception](https://mlsanity.com/paper/2510.12720/) (2025)
- [VABench: A Comprehensive Benchmark for Audio-Video Generation](https://mlsanity.com/paper/2512.09299/) (2025)
- [AVoCaDO: An Audiovisual Video Captioner Driven by Temporal Orchestration](https://mlsanity.com/paper/2510.10395/) (2025)
- [LongCat-Flash-Omni Technical Report](https://mlsanity.com/paper/2511.00279/) (2025)

## Abstract (arXiv)

Detailed video captioning has become an essential semantic interface for multi-modal understanding and generation. While existing methods mainly target short videos, extending them to long videos remains challenging, primarily due to limited context windows of current video understanding models. A naive solution is to adopt segment-wise captioning followed by concatenation, yet this paradigm often suffers from inconsistent subject references and fragmented long-range narratives. To mitigate these issues, we introduce PIGEONS, an incremental long-video captioning model built upon a Persistent Identity Gallery and an Evolving Overarching Narrative Summary. Before caption generation, PIGEONS scans the entire video to construct an identity gallery that captures key subjects along with their visual references, thereby providing a stable referential anchor. The model then performs clip-by-clip captioning conditioned on the persistent gallery and an evolving overarching narrative summary, which compactly maintains and updates essential long-range contextual information after each captioning step. To enable these capabilities, we develop a three-stage training strategy that integrates supervised fine-tuning and reinforcement learning. Extensive experiments demonstrate that PIGEONS markedly improves long-video captioning performance, achieving superior accuracy, completeness, and subject referential consistency over existing methods, while maintaining competitive performance on short-video captioning.