mlsanityData to 14 Oct 2026

Hierarchical3D Adapters for Long Video-to-text Summarization

arXiv 2210.04829 ↗First posted 10 Oct 2022cs.CLcited by 12

Not on the map. Why?

Abstract (arXiv)

In this paper, we focus on video-to-text summarization and investigate how to best utilize multimodal information for summarizing long inputs (e.g., an hour-long TV show) into long outputs (e.g., a multi-sentence summary). We extend SummScreen (Chen et al., 2021), a dialogue summarization dataset consisting of transcripts of TV episodes with reference summaries, and create a multimodal variant by collecting corresponding full-length videos. We incorporate multimodal information into a pre-trained textual summarizer efficiently using adapter modules augmented with a hierarchical structure while tuning only 3.8% of model parameters. Our experiments demonstrate that multimodal information offers superior performance over more memory-heavy and fully fine-tuned textual summarization methods.

Key facts. Cited by 12 papers in the map (data through 14 Oct 2026). Builds on 59 of its 88 references that are in the map. First posted on arXiv 10 Oct 2022.

Builds on

59 of 88 references.

References · 59

  1. 2017Attention Is All You Needcited by 60,361
  2. 2014Adam: A Method for Stochastic Optimizationcited by 37,240
  3. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,744
  4. 2020Language Models are Few-Shot Learnerscited by 32,419
  5. 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,599
  6. 2019RoBERTa: A Robustly Optimized BERT Pretraining Approachcited by 13,372
  7. 2019Sentence-BERT: Sentence Embeddings using Siamese BERT-Networkscited by 9,170
  8. 2019BART: Denoising Sequence-to-Sequence Pre-training for Natural Language Generation, Translation, and Comprehensioncited by 5,893
  9. 2019BERTScore: Evaluating Text Generation with BERTcited by 5,512
  10. 2015Rethinking the Inception Architecture for Computer Visioncited by 5,475
  11. 2019Parameter-Efficient Transfer Learning for NLPcited by 4,236
  12. 2016SQuAD: 100,000+ Questions for Machine Comprehension of Textcited by 4,148
38 more
  1. 2021Prefix-Tuning: Optimizing Continuous Prompts for Generationcited by 3,915
  2. 2020Longformer: The Long-Document Transformercited by 3,413
  3. 2017A Broad-Coverage Challenge Corpus for Sentence Understanding through Inferencecited by 2,682
  4. 2018SlowFast Networks for Video Recognitioncited by 1,867
  5. 2020Big Bird: Transformers for Longer Sequencescited by 1,684
  6. 2020Reformer: The Efficient Transformercited by 1,668
  7. 2018Don't Give Me the Details, Just the Summary! Topic-Aware Convolutional Neural Networks for Extreme Summarizationcited by 1,317
  8. 2017Get To The Point: Summarization with Pointer-Generator Networkscited by 1,292
  9. 2019PEGASUS: Pre-training with Extracted Gap-sentences for Abstractive Summarizationcited by 1,102
  10. 2020On Faithfulness and Factuality in Abstractive Summarizationcited by 900
  11. 2016Abstractive Text Summarization Using Sequence-to-Sequence RNNs and Beyondcited by 812
  12. 2021Multimodal Few-Shot Learning with Frozen Language Modelscited by 661
  13. 2017Towards Automatic Learning of Procedures from Web Instructional Videoscited by 619
  14. 2019Evaluating the Factual Consistency of Abstractive Text Summarizationcited by 584
  15. 2017Learning multiple visual domains with residual adapterscited by 557
  16. 2019Text Summarization with Pretrained Encoderscited by 556
  17. 2019SAMSum Corpus: A Human-annotated Dialogue Dataset for Abstractive Summarizationcited by 535
  18. 2017A Deep Reinforced Model for Abstractive Summarizationcited by 365
  19. 2021QMSum: A New Benchmark for Query-based Multi-domain Meeting Summarizationcited by 365
  20. 2020HERO: Hierarchical Encoder for Video+Language Omni-representation Pre-trainingcited by 362
  21. 2021VL-Adapter: Parameter-Efficient Transfer Learning for Vision-and-Language Taskscited by 320
  22. 2020UniVL: A Unified Video and Language Pre-Training Model for Multimodal Understanding and Generationcited by 259
  23. 2020Sparse Sinkhorn Attentioncited by 251
  24. 2020TVR: A Large-Scale Dataset for Video-Subtitle Moment Retrievalcited by 215
  25. 2016Movie Descriptioncited by 181
  26. 2018Bottom-Up Abstractive Summarizationcited by 177
  27. 2022SCROLLS: Standardized CompaRison Over Long Language Sequencescited by 170
  28. 2018How2: A Large-scale Dataset for Multimodal Language Understandingcited by 138
  29. 2019Hierarchical Transformers for Multi-Document Summarizationcited by 132
  30. 2021Q2: Evaluating Factual Consistency in Knowledge-Grounded Dialogues via Question Generation and Question Answeringcited by 132
  31. 2022Understanding Factual Errors in Summarization: Errors, Summarizers, Datasets, Error Detectorscited by 104
  32. 2021VLM: Task-agnostic Video-Language Model Pre-training for Video Understandingcited by 102
  33. 2021VALUE: A Multi-Task Benchmark for Video-and-Language Understanding Evaluationcited by 95
  34. 2020MART: Memory-Augmented Recurrent Transformer for Coherent Video Paragraph Captioningcited by 79
  35. 2021SummScreen: A Dataset for Abstractive Screenplay Summarizationcited by 79
  36. 2021DialogLM: Pre-trained Model for Long Dialogue Understanding and Summarizationcited by 72
  37. 2021Summ^N: A Multi-Stage Summarization Framework for Long Input Dialogues and Documentscited by 65
  38. 2014Coherent Multi-Sentence Video Description with Variable Level of Detailcited by 45

50 most cited of 59 All 59 (JSON)

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2023: 2; 2024: 3; 2025: 5; 2026: 2.

12 papers.

  1. 2023Modular Deep Learningcited by 83
  2. 2023Visual Tuningcited by 34
  3. 2024A Modular Approach for Multimodal Summarization of TV Showscited by 13
  4. 2025What Is That Talk About? A Video-to-Text Summarization Dataset for Scientific Presentationscited by 11
  5. 2024ScreenWriter: Automatic Screenplay Generation and Movie Summarisationcited by 5
  6. 2025Parameter-free Video Segmentation for Vision and Language Understandingcited by 3
  7. 2026Cut to the Chase: Training-free Multimodal Summarization via Chain-of-Eventscited by 3
  8. 2025Unlocking Financial Insights: An advanced Multimodal Summarization with Multimodal Output Framework for Financial Advisory Videoscited by 2
  9. 2024"Previously on ..." From Recaps to Story Summarizationcited by 1
  10. 2025Integrating Video and Text: A Balanced Approach to Multimodal Summary Generation and Evaluationcited by 0
  11. 2025Minimal Clips, Maximum Salience: Long Video Summarization via Key Moment Extractioncited by 0
  12. 2026Summarization is Not Dead Yetcited by 0