mlsanityData to 7 Oct 2026

COIN: A Large-scale Dataset for Comprehensive Instructional Video Analysis

arXiv 1903.02874 ↗First posted 7 Mar 2019cs.CVcited by 316

Not on the map. Why?

Cited by 46 in Multimodal LLMs / vision-language models, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

There are substantial instructional videos on the Internet, which enables us to acquire knowledge for completing various tasks. However, most existing datasets for instructional video analysis have the limitations in diversity and scale,which makes them far from many real-world applications where more diverse activities occur. Moreover, it still remains a great challenge to organize and harness such data. To address these problems, we introduce a large-scale dataset called "COIN" for COmprehensive INstructional video analysis. Organized with a hierarchical structure, the COIN dataset contains 11,827 videos of 180 tasks in 12 domains (e.g., vehicles, gadgets, etc.) related to our daily life. With a new developed toolbox, all the videos are annotated effectively with a series of step descriptions and the corresponding temporal boundaries. Furthermore, we propose a simple yet effective method to capture the dependencies among different steps, which can be easily plugged into conventional proposal-based action detection methods for localizing important steps in instructional videos. In order to provide a benchmark for instructional video analysis, we evaluate plenty of approaches on the COIN dataset under different evaluation criteria. We expect the introduction of the COIN dataset will promote the future in-depth research on instructional video analysis for the community.

Key facts. Cited by 316 papers in the map (data through 7 Oct 2026). First posted on arXiv 7 Mar 2019.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 21; 2022: 32; 2023: 51; 2024: 74; 2025: 81; 2026: 57.

316 papers.

Sort:

By area:

  1. 2025V-JEPA 2: Self-Supervised Video Models Enable Understanding, Prediction and Planningcited by 738
  2. 2021VideoCLIP: Contrastive Pre-training for Zero-shot Video-Text Understandingcited by 596
  3. 2025VideoLLaMA 3: Frontier Multimodal Foundation Models for Image and Video Understandingcited by 502
  4. 2023TimeChat: A Time-sensitive Multimodal Large Language Model for Long Video Understandingcited by 399
  5. 2024VideoMamba: State Space Model for Efficient Video Understandingcited by 325
  6. 2022Assembly101: A Large-Scale Multi-View Video Dataset for Understanding Procedural Activitiescited by 284
  7. 2024MA-LMM: Memory-Augmented Large Multimodal Model for Long-Term Video Understandingcited by 223
  8. 2024VideoLLM-online: Online Video Large Language Model for Streaming Videocited by 216
  9. 2024Tarsier: Recipes for Training and Evaluating Large Video Description Modelscited by 140
  10. 2025OVO-Bench: How Far is Your Video-LLMs from Real-World Online Video Understanding?cited by 134
  11. 2024LITA: Language Instructed Temporal-Localization Assistantcited by 124
  12. 2022EgoTaskQA: Understanding Human Tasks in Egocentric Videoscited by 112
38 more
  1. 2022Long Movie Clip Classification with State-Space Video Modelscited by 106
  2. 2022FineDiving: A Fine-grained Dataset for Procedure-aware Action Quality Assessmentcited by 106
  3. 2023Selective Structured State-Spaces for Long-Form Video Understandingcited by 105
  4. 2024VideoPrism: A Foundational Visual Encoder for Video Understandingcited by 103
  5. 2021VLM: Task-agnostic Video-Language Model Pre-training for Video Understandingcited by 102
  6. 2021TACo: Token-aware Cascade Contrastive Learning for Video-Text Alignmentcited by 102
  7. 2024TimeSuite: Improving MLLMs for Long Video Understanding via Grounded Tuningcited by 99
  8. 2024TemporalBench: Benchmarking Fine-grained Temporal Understanding for Multimodal Video Modelscited by 96
  9. 2025TimeChat-Online: 80% Visual Tokens are Naturally Redundant in Streaming Videoscited by 90
  10. 2024Video ReCap: Recursive Captioning of Hour-Long Videoscited by 88
  11. 2024VideoHallucer: Evaluating Intrinsic and Extrinsic Hallucinations in Large Video-Language Modelscited by 88
  12. 2024TRACE: Temporal Grounding Video LLM via Causal Event Modelingcited by 87
  13. 2022Learning To Recognize Procedural Activities with Distant Supervisioncited by 85
  14. 2022Self-Supervised Learning for Videos: A Surveycited by 85
  15. 2022Temporal Action Segmentation: An Analysis of Modern Techniquescited by 85
  16. 2023Hierarchical Video-Moment Retrieval and Step-Captioningcited by 83
  17. 2022My View is the Best View: Procedure Learning from Egocentric Videoscited by 81
  18. 2023AntGPT: Can Large Language Models Help Long-term Action Anticipation from Videos?cited by 79
  19. 2021Group-aware Contrastive Regression for Action Quality Assessmentcited by 75
  20. 2025StreamForest: Efficient Online Video Understanding with Persistent Event Memorycited by 71
  21. 2023Learning Multi-modal Representations by Watching Hundreds of Surgical Video Lecturescited by 68
  22. 2024E.T. Bench: Towards Open-Ended Event-Level Video-Language Understandingcited by 68
  23. 2023HierVL: Learning Hierarchical Video-Language Embeddingscited by 66
  24. 2023UnLoc: A Unified Framework for Video Localization Taskscited by 66
  25. 2023EgoPlan-Bench: Benchmarking Multimodal Large Language Models for Human-Level Planningcited by 66
  26. 2024Mamba-360: Survey of State Space Models as Transformer Alternative for Long Sequence Modelling: Methods, Applications, and Challengescited by 63
  27. 2025Eagle 2.5: Boosting Long-Context Post-Training for Frontier Vision-Language Modelscited by 62
  28. 2025VL-JEPA: Joint Embedding Predictive Architecture for Vision-languagecited by 61
  29. 2021Visual Semantic Role Labeling for Video Understandingcited by 60
  30. 2022Long-Form Video-Language Pre-Training with Multimodal Temporal Contrastive Learningcited by 59
  31. 2024Online Video Understanding: OVBench and VideoChat-Onlinecited by 58
  32. 2025LLaVA-ST: A Multimodal Large Language Model for Fine-Grained Spatial-Temporal Understandingcited by 57
  33. 2025StreamBridge: Turning Your Offline Video Large Language Model into a Proactive Streaming Assistantcited by 56
  34. 2024VideoLLM-MoD: Efficient Video-Language Streaming with Mixture-of-Depths Vision Computationcited by 55
  35. 2021Home Action Genome: Cooperative Compositional Action Understandingcited by 52
  36. 2022P3IV: Probabilistic Procedure Planning from Instructional Videos with Weak Supervisioncited by 52
  37. 2022Efficient Movie Scene Detection using State-Space Transformerscited by 51
  38. 2022Bridge-Prompt: Towards Ordinal Action Understanding in Instructional Videoscited by 50

50 most cited of 316 All 316 (JSON)