mlsanityData to 14 Oct 2026

Oops! Predicting Unintentional Action in Video

arXiv 1911.11206 ↗First posted 25 Nov 2019cs.CV cs.LG eess.IVcited by 82

Not on the map. Why?

Abstract (arXiv)

From just a short glance at a video, we can often tell whether a person's action is intentional or not. Can we train a model to recognize this? We introduce a dataset of in-the-wild videos of unintentional action, as well as a suite of tasks for recognizing, localizing, and anticipating its onset. We train a supervised neural network as a baseline and analyze its performance compared to human consistency on the tasks. We also investigate self-supervised representations that leverage natural signals in our dataset, and show the effectiveness of an approach that uses the intrinsic speed of video to perform competitively with highly-supervised pretraining. However, a significant gap between machine and human performance remains. The project website is available at https://oops.cs.columbia.edu

Key facts. Cited by 82 papers in the map (data through 14 Oct 2026). First posted on arXiv 25 Nov 2019.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2021: 14; 2022: 9; 2023: 12; 2024: 16; 2025: 18; 2026: 13.

82 papers.

Sort:

By area:

  1. 2021Ego4D: Around the World in 3,000 Hours of Egocentric Videocited by 1,556
  2. 2024Tarsier: Recipes for Training and Evaluating Large Video Description Modelscited by 141
  3. 2024TemporalBench: Benchmarking Fine-grained Temporal Understanding for Multimodal Video Modelscited by 97
  4. 2025Eagle 2.5: Boosting Long-Context Post-Training for Frontier Vision-Language Modelscited by 64
  5. 2021VIMPAC: Video Pre-Training via Masked Token Prediction and Contrastive Learningcited by 56
  6. 2025IntPhys 2: Benchmarking Intuitive Physics Understanding In Complex Synthetic Environmentscited by 52
  7. 2021Time-Equivariant Contrastive Video Representation Learningcited by 44
  8. 2025ViSpeak: Visual Instruction Feedback in Streaming Videoscited by 42
  9. 2024Mind the Time: Temporally-Controlled Multi-Event Video Generationcited by 40
  10. 2023Learning Object State Changes in Videos: An Open-World Perspectivecited by 36
  11. 2023FunQA: Towards Surprising Video Comprehensioncited by 35
  12. 2021Action Shuffle Alternating Learning for Unsupervised Action Segmentationcited by 33
38 more
  1. 2025Mavors: Multi-granularity Video Representation for Multimodal Large Language Modelcited by 32
  2. 2021Space-Time Crop & Attend: Improving Cross-modal Video Representation Learningcited by 31
  3. 2024Universal Time-Series Representation Learning: A Surveycited by 30
  4. 2023StoryBench: A Multifaceted Benchmark for Continuous Story Visualizationcited by 29
  5. 2023Connecting Vision and Language with Video Localized Narrativescited by 24
  6. 2022PedFormer: Pedestrian Behavior Prediction via Cross-Modal Attention Modulation and Gated Multitask Learningcited by 23
  7. 2024FinePOSE: Fine-Grained Prompt-Driven 3D Human Pose Estimation via Diffusion Modelscited by 23
  8. 2021Learning Representational Invariances for Data-Efficient Action Recognitioncited by 22
  9. 2024Distilling Vision-Language Models on Millions of Videoscited by 20
  10. 2023PECoP: Parameter Efficient Continual Pretraining for Action Quality Assessmentcited by 19
  11. 2023Audio-Visual Contrastive Learning with Temporal Self-Supervisioncited by 18
  12. 2023Every Mistake Counts in Assemblycited by 18
  13. 2023Permutation-Aware Action Segmentation via Unsupervised Frame-to-Segment Alignmentcited by 17
  14. 2024ViCaS: A Dataset for Combining Holistic and Pixel-level Video Understanding using Captions with Grounded Segmentationcited by 17
  15. 2021SSCAP: Self-supervised Co-occurrence Action Parsing for Unsupervised Temporal Action Segmentationcited by 15
  16. 2021Hallucinating Pose-Compatible Scenescited by 14
  17. 2023A Large-scale Study of Spatiotemporal Representation Learning with a New Benchmark on Action Recognitioncited by 14
  18. 2023STEPs: Self-Supervised Key Step Extraction and Localization from Unlabeled Procedural Videoscited by 13
  19. 2025Reasoning Segmentation for Images and Videos: A Surveycited by 12
  20. 2021Deep Feedback Inverse Problem Solvercited by 11
  21. 2024Beyond Walking: A Large-Scale Image-Text Benchmark for Text-based Person Anomaly Searchcited by 11
  22. 2022How Would The Viewer Feel? Estimating Wellbeing From Video Scenarioscited by 9
  23. 2024Human Action Anticipation: A Surveycited by 9
  24. 2024Black Swan: Abductive and Defeasible Video Reasoning in Unpredictable Eventscited by 9
  25. 2023ACQUIRED: A Dataset for Answering Counterfactual Questions In Real-Life Videoscited by 8
  26. 2024StimuVAR: Spatiotemporal Stimuli-aware Video Affective Reasoning with Multimodal Large Language Modelscited by 8
  27. 2022Self-supervised Learning for Unintentional Action Predictioncited by 7
  28. 2026IPIBench: Evaluating Interactive Proactive Intelligence of MLLMs under Continuous Streamscited by 7
  29. 2023Multi-Modal Gaze Following in Conversational Scenarioscited by 6
  30. 2024Deep Learning for Computer Vision based Activity Recognition and Fall Detection of the Elderly: a Systematic Reviewcited by 6
  31. 2026ROMA: Real-time Omni-Multimodal Assistant with Interactive Streaming Understandingcited by 6
  32. 2021Review of Video Predictive Understanding: Early Action Recognition and Future Action Predictioncited by 5
  33. 2024Point-VOS: Pointing Up Video Object Segmentationcited by 5
  34. 2025Building Egocentric Procedural AI Assistant: Methods, Benchmarks, and Challengescited by 5
  35. 2026One Token per Highly Selective Frame: Towards Extreme Compression for Long Video Understandingcited by 5
  36. 2021InDiD: Instant Disorder Detection via Representation Learningcited by 4
  37. 2025ImageChain: Advancing Sequential Image-to-Text Reasoning in Multimodal Large Language Modelscited by 4
  38. 2021Auxiliary Learning for Self-Supervised Video Representation via Similarity-based Knowledge Distillationcited by 3

50 most cited of 82 All 82 (JSON)