mlsanityData to 7 Oct 2026

A Dataset for Medical Instructional Video Classification and Question Answering

arXiv 2201.12888 ↗First posted 30 Jan 2022cs.CV cs.CLcited by 26

Not on the map. Why?

Abstract (arXiv)

This paper introduces a new challenge and datasets to foster research toward designing systems that can understand medical videos and provide visual answers to natural language questions. We believe medical videos may provide the best possible answers to many first aids, medical emergency, and medical education questions. Toward this, we created the MedVidCL and MedVidQA datasets and introduce the tasks of Medical Video Classification (MVC) and Medical Visual Answer Localization (MVAL), two tasks that focus on cross-modal (medical language and medical video) understanding. The proposed tasks and datasets have the potential to support the development of sophisticated downstream applications that can benefit the public and medical practitioners. Our datasets consist of 6,117 annotated videos for the MVC task and 3,010 annotated questions and answers timestamps from 899 videos for the MVAL task. These datasets have been verified and corrected by medical informatics experts. We have also benchmarked each task with the created MedVidCL and MedVidQA datasets and proposed the multimodal learning methods that set competitive baselines for future research.

Key facts. Cited by 26 papers in the map (data through 7 Oct 2026). Builds on 22 of its 40 references that are in the map. First posted on arXiv 30 Jan 2022.

Builds on

22 of 40 references.

References · 22

  1. 2017Attention Is All You Needcited by 59,932
  2. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  3. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 29,915
  4. 2019RoBERTa: A Robustly Optimized BERT Pretraining Approachcited by 13,266
  5. 2017Quo Vadis, Action Recognition? A New Model and the Kinetics Datasetcited by 3,259
  6. 2020Big Bird: Transformers for Longer Sequencescited by 1,673
  7. 2020Oscar: Object-Semantics Aligned Pre-training for Vision-Language Taskscited by 1,213
  8. 2014Convolutional Neural Networks for Sentence Classificationcited by 1,124
  9. 2019HowTo100M: Learning a Text-Video Embedding by Watching Hundred Million Narrated Video Clipscited by 946
  10. 2017Localizing Moments in Video with Natural Languagecited by 660
  11. 2018TVQA: Localized, Compositional Video Question Answeringcited by 463
  12. 2017Population Based Training of Neural Networkscited by 351
10 more
  1. 2016YouTube-8M: A Large-Scale Video Classification Benchmarkcited by 324
  2. 2015MovieQA: Understanding Stories in Movies through Question-Answeringcited by 304
  3. 2016Bidirectional Attention Flow for Machine Comprehensioncited by 278
  4. 2020UniVL: A Unified Video and Language Pre-Training Model for Multimodal Understanding and Generationcited by 258
  5. 2020Span-based Localizing Network for Natural Language Video Localizationcited by 234
  6. 2018To Find Where You Talk: Temporal Sentence Localization in Video with Attention Based Location Regressioncited by 157
  7. 2019TVQA+: Spatio-Temporal Grounding for Video Question Answeringcited by 153
  8. 2018Localizing Moments in Video with Temporal Languagecited by 83
  9. 2016MarioQA: Answering Questions by Watching Gameplay Videoscited by 42
  10. 2018Multi-Label Image Classification via Knowledge Distillation from Weakly-Supervised Detectioncited by 37

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2022: 5; 2023: 2; 2024: 5; 2025: 10; 2026: 4.

26 papers.

Sort:

By area:

  1. 2025MotionBench: Benchmarking and Improving Fine-grained Video Motion Understanding for Vision Language Modelscited by 69
  2. 2025Eagle 2.5: Boosting Long-Context Post-Training for Frontier Vision-Language Modelscited by 62
  3. 2022Towards Visual-Prompt Temporal Answering Grounding in Medical Instructional Videocited by 43
  4. 2024FineParser: A Fine-grained Spatio-temporal Action Parser for Human-centric Action Quality Assessmentcited by 24
  5. 2023NurViD: A Large Expert-Level Video Database for Nursing Procedure Activity Understandingcited by 20
  6. 2024Large Language Model Benchmarks in Medical Taskscited by 16
  7. 2025Cancer-Myth: Evaluating Large Language Models on Patient Questions with False Presuppositionscited by 15
  8. 2022Learning to Locate Visual Answer in Video Corpus Using Questioncited by 10
  9. 2022Visual Answer Localization with Cross-modal Mutual Knowledge Transfercited by 7
  10. 2025Can Vision-Language Models Answer Face to Face Questions in the Real-World?cited by 7
  11. 2025From large language models to multimodal AI: A scoping review on the potential of generative AI in medicinecited by 5
  12. 2025How Well Can General Vision-Language Models Learn Medicine By Watching Public Educational Videos?cited by 5
14 more
  1. 2022Medical Image Retrieval via Nearest Neighbor Search on Pre-trained Image Featurescited by 3
  2. 2025EyePCR: A Comprehensive Benchmark for Fine-Grained Perception, Knowledge Comprehension and Clinical Reasoning in Ophthalmic Surgerycited by 3
  3. 2022Human in the loop approaches in multi-modal conversational task guidance system developmentcited by 2
  4. 2024TVR-Ranking: A Dataset for Ranked Video Moment Retrieval with Imprecise Queriescited by 2
  5. 2024Overview of TREC 2024 Medical Video Question Answering (MedVidQA) Trackcited by 2
  6. 2025Ask2Loc: Learning to Locate Instructional Visual Answers by Asking Questionscited by 2
  7. 2025Med-CRAFT: An Information System for Explainable and Configurable Construction of Multimodal Medical QA Datasetscited by 2
  8. 2023Towards Answering Health-related Questions from Medical Videos: Datasets and Approachescited by 1
  9. 2025Enhancing the Learning Experience: Using Vision-Language Models to Generate Questions for Educational Videoscited by 1
  10. 2026Overview of the NLPCC 2026 Shared Task 1: Difficulty-Aware Multilingual and Multimodal Medical Instructional Video Understanding Evaluationcited by 1
  11. 2024Multimodal Transformer With a Low-Computational-Cost Guaranteecited by 0
  12. 2026Seizure-Semiology-Suite (S3): A Clinically Multimodal Dataset, Benchmark, and Models for Seizure Semiology Understandingcited by 0
  13. 2026DAEP: Difficulty-Aware Evidence Planning for Medical Video Corpus Temporal Answer Groundingcited by 0
  14. 2026DynamicDx: Evaluating Evidence Acquisition in Video-Based Diagnosiscited by 0