mlsanityData to 7 Oct 2026

RareAct: A video dataset of unusual interactions

arXiv 2008.01018 ↗First posted 3 Aug 2020cs.CVcited by 23

Not on the map. Why?

Abstract (arXiv)

This paper introduces a manually annotated video dataset of unusual actions, namely RareAct, including actions such as "blend phone", "cut keyboard" and "microwave shoes". RareAct aims at evaluating the zero-shot and few-shot compositionality of action recognition models for unlikely compositions of common action verbs and object nouns. It contains 122 different actions which were obtained by combining verbs and nouns rarely co-occurring together in the large-scale textual corpus from HowTo100M, but that frequently appear separately. We provide benchmarks using a state-of-the-art HowTo100M pretrained video and text model and show that zero-shot and few-shot compositionality of actions remains a challenging and unsolved task.

Key facts. Cited by 23 papers in the map (data through 7 Oct 2026). First posted on arXiv 3 Aug 2020.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 1; 2022: 2; 2023: 7; 2024: 6; 2025: 5; 2026: 2.

23 papers.

Sort:

By area:

  1. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
  2. 2022Flamingo: a Visual Language Model for Few-Shot Learningcited by 4,337
  3. 2023Vision-Language Models for Vision Tasks: A Surveycited by 834
  4. 2024Tarsier: Recipes for Training and Evaluating Large Video Description Modelscited by 140
  5. 2025Tarsier2: Advancing Large Vision-Language Models from Detailed Video Description to Comprehensive Video Understandingcited by 80
  6. 2023DeAR: Debiasing Vision-Language Models with Additive Residualscited by 70
  7. 2022Look for the Change: Learning Object States and State-Modifying Actions from Untrimmed Web Videoscited by 39
  8. 2023ViLMA: A Zero-Shot Benchmark for Linguistic and Temporal Grounding in Video-Language Modelscited by 26
  9. 2023Masked Modeling for Self-supervised Representation Learning on Vision and Beyondcited by 13
  10. 2023CDFSL-V: Cross-Domain Few-Shot Learning for Videoscited by 12
  11. 2024CLoVe: Encoding Compositional Language in Contrastive Vision-Language Modelscited by 11
  12. 2023DEVIAS: Learning Disentangled Video Representations of Action and Scenecited by 7
11 more
  1. 2025VaseVQA: Multimodal Agent and Benchmark for Ancient Greek Potterycited by 6
  2. 2024Open Vocabulary Multi-Label Video Classificationcited by 5
  3. 2025LEGO Co-builder: Exploring Fine-Grained Vision-Language Modeling for Multimodal LEGO Assembly Assistantscited by 5
  4. 2023Uncovering the Hidden Dynamics of Video Self-supervised Learning under Distribution Shiftscited by 4
  5. 2024DMSD-CDFSAR: Distillation from Mixed-Source Domain for Cross-Domain Few-shot Action Recognitioncited by 3
  6. 2024TAMT: Temporal-Aware Model Tuning for Cross-Domain Few-Shot Action Recognitioncited by 3
  7. 2025Safeguarding Vision-Language Models: Mitigating Vulnerabilities to Gaussian Noise in Perturbation-based Attackscited by 3
  8. 2024A Comprehensive Methodological Survey of Human Activity Recognition Across Divers Data Modalitiescited by 2
  9. 2025Can Vision Language Models Understand Mimed Actions?cited by 2
  10. 2026Explainable Forensics of Manipulated Segments in Untrimmed Long Videoscited by 0
  11. 2026Gen2Balance: Generative Balancing for Long-Tailed Video Action Recognitioncited by 0