mlsanityData to 7 Oct 2026

WhyAct: Identifying Action Reasons in Lifestyle Vlogs

arXiv 2109.02747 ↗First posted 6 Sep 2021cs.CV cs.CLcited by 6

Not on the map. Why?

Abstract (arXiv)

We aim to automatically identify human action reasons in online videos. We focus on the widespread genre of lifestyle vlogs, in which people perform actions while verbally describing them. We introduce and make publicly available the WhyAct dataset, consisting of 1,077 visual actions manually annotated with their reasons. We describe a multimodal model that leverages visual and textual information to automatically infer the reasons corresponding to an action presented in the video.

Key facts. Cited by 6 papers in the map (data through 7 Oct 2026). Builds on 34 of its 58 references that are in the map. First posted on arXiv 6 Sep 2021.

Builds on

34 of 58 references.

References · 34

  1. 2017Attention Is All You Needcited by 59,932
  2. 2014Microsoft COCO: Common Objects in Contextcited by 17,498
  3. 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,491
  4. 2019Sentence-BERT: Sentence Embeddings using Siamese BERT-Networkscited by 9,049
  5. 2017Quo Vadis, Action Recognition? A New Model and the Kinetics Datasetcited by 3,259
  6. 2017A Broad-Coverage Challenge Corpus for Sentence Understanding through Inferencecited by 2,665
  7. 2018SlowFast Networks for Video Recognitioncited by 1,862
  8. 2017A Closer Look at Spatiotemporal Convolutions for Action Recognitioncited by 1,139
  9. 2016ConceptNet 5.5: An Open Multilingual Graph of General Knowledgecited by 1,065
  10. 2019HowTo100M: Learning a Text-Video Embedding by Watching Hundred Million Narrated Video Clipscited by 946
  11. 2018Scaling Egocentric Vision: The EPIC-KITCHENS Datasetcited by 789
  12. 2016CNN Architectures for Large-Scale Audio Classificationcited by 701
22 more
  1. 2017Towards Automatic Learning of Procedures from Web Instructional Videoscited by 616
  2. 2018From Recognition to Cognition: Visual Commonsense Reasoningcited by 553
  3. 2016Hollywood in Homes: Crowdsourcing Data Collection for Activity Understandingcited by 535
  4. 2016YouTube-8M: A Large-Scale Video Classification Benchmarkcited by 324
  5. 2018Video Action Transformer Networkcited by 316
  6. 2019COIN: A Large-scale Dataset for Comprehensive Instructional Video Analysiscited by 316
  7. 2019Benchmarking Zero-shot Text Classification: Datasets, Evaluation and Entailment Approachcited by 299
  8. 2020COMET-ATOMIC 2020: On Symbolic and Neural Commonsense Knowledge Graphscited by 254
  9. 2018Human Action Recognition and Prediction: A Surveycited by 226
  10. 2018Rethinking the Faster R-CNN Architecture for Temporal Action Localizationcited by 194
  11. 2017CDC: Convolutional-De-Convolutional Networks for Precise Temporal Action Localization in Untrimmed Videoscited by 112
  12. 2020GLUCOSE: GeneraLized and COntextualized Story Explanationscited by 85
  13. 2020A Better Use of Audio-Visual Cues: Dense Video Captioning with Bi-modal Transformercited by 75
  14. 2017From Lifestyle Vlogs to Everyday Interactionscited by 42
  15. 2020Video2Commonsense: Generating Commonsense Descriptions to Enrich Video Captioningcited by 36
  16. 2017What Actions are Needed for Understanding Human Actions in Videos?cited by 29
  17. 2020Intentonomy: a Dataset and Study towards Human Intent Understandingcited by 22
  18. 2020KVL-BERT: Knowledge Enhanced Visual-and-Linguistic BERT for Visual Commonsense Reasoningcited by 18
  19. 2018A Span Selection Model for Semantic Role Labelingcited by 15
  20. 2019Identifying Visible Actions in Lifestyle Vlogscited by 6
  21. 2020Learning Contextual Causality from Time-consecutive Imagescited by 4
  22. 2020Adding Knowledge to Unsupervised Algorithms for the Recognition of Intentcited by 2

Built on by

Cited by year

Citing papers per year: 2022: 2; 2023: 3; 2024: 1.

6 papers.

  1. 2022The Abduction of Sherlock Holmes: A Dataset for Visual Abductive Reasoningcited by 38
  2. 2023Has It All Been Solved? Open NLP Research Questions Not Solved by Large Language Modelscited by 7
  3. 2023Personality-aware Human-centric Multimodal Reasoning: A New Task, Dataset and Baselinescited by 2
  4. 2022VScript: Controllable Script Generation with Visual Presentationcited by 1
  5. 2024MEEL: Multi-Modal Event Evolution Learningcited by 1
  6. 2023Human Action Co-occurrence in Lifestyle Vlogs using Graph Link Predictioncited by 0