# NarraScore: Bridging Visual Narrative and Musical Dynamics via Hierarchical Affective Control

arXiv: https://arxiv.org/abs/2602.09070 · HTML: https://mlsanity.com/paper/2602.09070/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 28 of its 47 references that are in the map.
- First posted on arXiv 9 Feb 2026.

## Builds on (most cited first, 28 of 28)

- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [Gemini 2.5: Pushing the Frontier with Advanced Reasoning, Multimodality, Long Context, and Next Generation Agentic Capabilities](https://mlsanity.com/paper/2507.06261/) (2025)
- [SlowFast Networks for Video Recognition](https://mlsanity.com/paper/1812.03982/) (2018)
- [An Empirical Evaluation of Generic Convolutional and Recurrent Networks for Sequence Modeling](https://mlsanity.com/paper/1803.01271/) (2018)
- [Video-LLaMA: An Instruction-tuned Audio-Visual Language Model for Video Understanding](https://mlsanity.com/paper/2306.02858/) (2023)
- [VideoMAE: Masked Autoencoders are Data-Efficient Learners for Self-Supervised Video Pre-Training](https://mlsanity.com/paper/2203.12602/) (2022)
- [VideoLLaMA 2: Advancing Spatial-Temporal Modeling and Audio Understanding in Video-LLMs](https://mlsanity.com/paper/2406.07476/) (2024)
- [VideoMAE V2: Scaling Video Masked Autoencoders with Dual Masking](https://mlsanity.com/paper/2303.16727/) (2023)
- [MusicLM: Generating Music From Text](https://mlsanity.com/paper/2301.11325/) (2023)
- [Improving Video Generation with Human Feedback](https://mlsanity.com/paper/2501.13918/) (2025)
- [Video models are zero-shot learners and reasoners](https://mlsanity.com/paper/2509.20328/) (2025)
- [AVLnet: Learning Audio-Visual Language Representations from Instructional Videos](https://mlsanity.com/paper/2006.09199/) (2020)
- [Music Source Separation in the Waveform Domain](https://mlsanity.com/paper/1911.13254/) (2019)
- [Foley Music: Learning to Generate Music from Videos](https://mlsanity.com/paper/2007.10984/) (2020)
- [Aff-Wild2: Extending the Aff-Wild Database for Affect Recognition](https://mlsanity.com/paper/1811.07770/) (2018)
- [Video Background Music Generation with Controllable Music Transformer](https://mlsanity.com/paper/2111.08380/) (2021)
- [Video2Music: Suitable Music Generation from Videos using an Affective Multimodal Transformer model](https://mlsanity.com/paper/2311.00968/) (2023)
- [Learning Emotion Representations from Verbal and Nonverbal Communication](https://mlsanity.com/paper/2305.13500/) (2023)
- [V2Meow: Meowing to the Visual Beat via Video-to-Music Generation](https://mlsanity.com/paper/2305.06594/) (2023)
- [VMAS: Video-to-Music Generation via Semantic Alignment in Web Music Videos](https://mlsanity.com/paper/2409.07450/) (2024)
- [MuVi: Video-to-Music Generation with Semantic Alignment and Rhythmic Synchronization](https://mlsanity.com/paper/2410.12957/) (2024)
- [VEATIC: Video-based Emotion and Affect Tracking in Context Dataset](https://mlsanity.com/paper/2309.06745/) (2023)
- [MusiConGen: Rhythm and Chord Control for Transformer-Based Text-to-Music Generation](https://mlsanity.com/paper/2407.15060/) (2024)
- [GVMGen: A General Video-to-Music Generation Model with Hierarchical Attentions](https://mlsanity.com/paper/2501.09972/) (2025)
- [A Comprehensive Survey on Generative AI for Video-to-Music Generation](https://mlsanity.com/paper/2502.12489/) (2025)
- [Controllable Video-to-Music Generation with Multiple Time-Varying Conditions](https://mlsanity.com/paper/2507.20627/) (2025)
- [Video Echoed in Music: Semantic, Temporal, and Rhythmic Alignment for Video-to-Music Generation](https://mlsanity.com/paper/2511.09585/) (2025)
- [Video Soundtrack Generation by Aligning Emotions and Temporal Boundaries](https://mlsanity.com/paper/2502.10154/) (2025)

## Abstract (arXiv)

Synthesizing coherent soundtracks for long-form videos remains a formidable challenge, currently stalled by three critical impediments: computational scalability, temporal coherence, and, most critically, a pervasive semantic blindness to evolving narrative logic. To bridge these gaps, we propose NarraScore, a hierarchical framework predicated on the core insight that emotion serves as a high-density compression of narrative logic. Uniquely, we repurpose frozen Vision-Language Models (VLMs) as continuous affective sensors, distilling high-dimensional visual streams into dense, narrative-aware Valence-Arousal trajectories. Mechanistically, NarraScore employs a Dual-Branch Injection strategy to reconcile global structure with local dynamism: a textitGlobal Semantic Anchor ensures stylistic stability, while a surgical textitToken-Level Affective Adapter modulates local tension via direct element-wise residual injection. This minimalist design bypasses the bottlenecks of dense attention and architectural cloning, effectively mitigating the overfitting risks associated with data scarcity. Experiments demonstrate that NarraScore achieves state-of-the-art consistency and narrative alignment with negligible computational overhead, establishing a fully autonomous paradigm for long-video soundtrack generation.