mlsanityData to 7 Oct 2026

ProBio: A Protocol-guided Multimodal Dataset for Molecular Biology Lab

arXiv 2311.00556 ↗First posted 1 Nov 2023cs.CVcited by 6

Not on the map. Why?

Abstract (arXiv)

The challenge of replicating research results has posed a significant impediment to the field of molecular biology. The advent of modern intelligent systems has led to notable progress in various domains. Consequently, we embarked on an investigation of intelligent monitoring systems as a means of tackling the issue of the reproducibility crisis. Specifically, we first curate a comprehensive multimodal dataset, named ProBio, as an initial step towards this objective. This dataset comprises fine-grained hierarchical annotations intended for the purpose of studying activity understanding in BioLab. Next, we devise two challenging benchmarks, transparent solution tracking and multimodal action recognition, to emphasize the unique characteristics and difficulties associated with activity understanding in BioLab settings. Finally, we provide a thorough experimental evaluation of contemporary video understanding models and highlight their limitations in this specialized domain to identify potential avenues for future research. We hope ProBio with associated benchmarks may garner increased focus on modern AI techniques in the realm of molecular biology.

Key facts. Cited by 6 papers in the map (data through 7 Oct 2026). Builds on 52 of its 109 references that are in the map. First posted on arXiv 1 Nov 2023.

Builds on

52 of 109 references.

References · 52

  1. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  2. 2014Adam: A Method for Stochastic Optimizationcited by 36,960
  3. 2019Sentence-BERT: Sentence Embeddings using Siamese BERT-Networkscited by 9,049
  4. 2023Segment Anythingcited by 8,158
  5. 2016Gaussian Error Linear Units (GELUs)cited by 3,431
  6. 2017Quo Vadis, Action Recognition? A New Model and the Kinetics Datasetcited by 3,259
  7. 2017The Kinetics Human Action Video Datasetcited by 1,894
  8. 2018SlowFast Networks for Video Recognitioncited by 1,862
  9. 2021Ego4D: Around the World in 3,000 Hours of Egocentric Videocited by 1,530
  10. 2018Datasheets for Datasetscited by 1,371
  11. 2017The "something something" video database for learning and evaluating visual common sensecited by 1,092
  12. 2016Realtime Multi-Person 2D Pose Estimation using Part Affinity Fieldscited by 1,061
38 more
  1. 2022YOLOv7: Trainable bag-of-freebies sets new state-of-the-art for real-time object detectorscited by 978
  2. 2019HowTo100M: Learning a Text-Video Embedding by Watching Hundred Million Narrated Video Clipscited by 946
  3. 2021Multiscale Vision Transformerscited by 896
  4. 2021NExT-QA:Next Phase of Question-Answering to Explaining Temporal Actionscited by 735
  5. 2021CLIP4Clip: An Empirical Study of CLIP for End to End Video Clip Retrievalcited by 645
  6. 2017Towards Automatic Learning of Procedures from Web Instructional Videoscited by 616
  7. 2021VideoCLIP: Contrastive Pre-training for Zero-shot Video-Text Understandingcited by 596
  8. 2019ActivityNet-QA: A Dataset for Understanding Complex Web Videos via Question Answeringcited by 559
  9. 2021MViTv2: Improved Multiscale Vision Transformers for Classification and Detectioncited by 516
  10. 2018TVQA: Localized, Compositional Video Question Answeringcited by 463
  11. 2018LaSOT: A High-quality Benchmark for Large-scale Single Object Trackingcited by 424
  12. 2019VATEX: A Large-Scale, High-Quality Multilingual Dataset for Video-and-Language Researchcited by 412
  13. 2021ActionCLIP: A New Paradigm for Video Action Recognitioncited by 317
  14. 2019COIN: A Large-scale Dataset for Comprehensive Instructional Video Analysiscited by 316
  15. 2021MERLOT: Multimodal Neural Script Knowledge Modelscited by 311
  16. 2018ATOM: Accurate Tracking by Overlap Maximizationcited by 285
  17. 2022Assembly101: A Large-Scale Multi-View Video Dataset for Understanding Procedural Activitiescited by 284
  18. 2020Just Ask: Learning to Answer Questions from Millions of Narrated Videoscited by 248
  19. 2023Diffusion-based Generation, Optimization, and Planning in 3D Scenescited by 237
  20. 2020FineGym: A Hierarchical Video Dataset for Fine-grained Action Understandingcited by 232
  21. 2018Moments in Time Dataset: one million videos for event understandingcited by 215
  22. 2019Cross-task weakly supervised learning from instructional videoscited by 199
  23. 2023Segment and Track Anythingcited by 193
  24. 2022Frozen CLIP Models are Efficient Video Learnerscited by 181
  25. 2022HUMANISE: Language-conditioned Human Motion Generation in 3D Scenescited by 156
  26. 2022Decoupling Features in Hierarchical Propagation for Video Object Segmentationcited by 146
  27. 2016TGIF: A New Dataset and Benchmark on Animated GIF Descriptioncited by 124
  28. 2021AGQA: A Benchmark for Compositional Spatio-Temporal Reasoningcited by 120
  29. 2023Explicit Visual Prompting for Low-Level Structure Segmentationscited by 113
  30. 2022EgoTaskQA: Understanding Human Tasks in Egocentric Videoscited by 112
  31. 2022FineDiving: A Fine-grained Dataset for Procedure-aware Action Quality Assessmentcited by 106
  32. 2020Segmenting Transparent Objects in the Wildcited by 98
  33. 2020The IKEA ASM Dataset: Understanding People Assembling Furniture through Actions, Objects and Posecited by 95
  34. 2023Vita-CLIP: Video and text adaptive CLIP via Multimodal Promptingcited by 90
  35. 2020Dark, Beyond Deep: A Paradigm Shift to Cognitive AI with Humanlike Common Sensecited by 62
  36. 2021Home Action Genome: Cooperative Compositional Action Understandingcited by 52
  37. 2020LEMMA: A Multi-view Dataset for Learning Multi-agent Multi-task Activitiescited by 46
  38. 2021YouRefIt: Embodied Reference Understanding with Language and Gesturecited by 32

50 most cited of 52 All 52 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 2; 2025: 3; 2026: 1.

6 papers.

  1. 2024AnySkill: Learning Open-Vocabulary Physical Skill for Interactive Agentscited by 32
  2. 2025Decompositional Neural Scene Reconstruction with Generative Diffusion Priorcited by 30
  3. 2025BioProBench: A Corpus and Benchmark for Biological Protocol Reasoning in Autonomous Sciencecited by 14
  4. 2025HumanPCR: Probing MLLM Capabilities in Diverse Human-Centric Scenescited by 2
  5. 2024BioVL-QR: Egocentric Biochemical Vision-and-Language Dataset Using Micro QR Codescited by 1
  6. 2026In-Context Model Predictive Generation: Open-Vocabulary Motion Synthesis from Language Models to Physicscited by 0