# Identifying Visible Actions in Lifestyle Vlogs

arXiv: https://arxiv.org/abs/1906.04236 · HTML: https://mlsanity.com/paper/1906.04236/

## Key facts

- Cited by 6 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 10 Jun 2019.

## Built on by (most cited first, 6 of 6)

- [TVLT: Textless Vision-Language Transformer](https://mlsanity.com/paper/2209.14156/) (2022)
- [Bridging the Digital Divide: Performance Variation across Socio-Economic Factors in Vision-Language Models](https://mlsanity.com/paper/2311.05746/) (2023)
- [Has It All Been Solved? Open NLP Research Questions Not Solved by Large Language Models](https://mlsanity.com/paper/2305.12544/) (2023)
- [WhyAct: Identifying Action Reasons in Lifestyle Vlogs](https://mlsanity.com/paper/2109.02747/) (2021)
- [When Did It Happen? Duration-informed Temporal Localization of Narrated Actions in Vlogs](https://mlsanity.com/paper/2202.08138/) (2022)
- [Human Action Co-occurrence in Lifestyle Vlogs using Graph Link Prediction](https://mlsanity.com/paper/2309.06219/) (2023)

## Abstract (arXiv)

We consider the task of identifying human actions visible in online videos. We focus on the widely spread genre of lifestyle vlogs, which consist of videos of people performing actions while verbally describing them. Our goal is to identify if actions mentioned in the speech description of a video are visually present. We construct a dataset with crowdsourced manual annotations of visible actions, and introduce a multimodal algorithm that leverages information derived from visual and linguistic clues to automatically infer which actions are visible in a video. We demonstrate that our multimodal algorithm outperforms algorithms based only on one modality at a time.