# HVM-1: Large-scale video models pretrained with nearly 5000 hours of human-like video data

arXiv: https://arxiv.org/abs/2407.18067 · HTML: https://mlsanity.com/paper/2407.18067/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 14 of its 25 references that are in the map.
- First posted on arXiv 25 Jul 2024.

## Builds on (most cited first, 14 of 14)

- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [ImageNet Large Scale Visual Recognition Challenge](https://mlsanity.com/paper/1409.0575/) (2014)
- [Masked Autoencoders Are Scalable Vision Learners](https://mlsanity.com/paper/2111.06377/) (2021)
- [FlashAttention-2: Faster Attention with Better Parallelism and Work Partitioning](https://mlsanity.com/paper/2307.08691/) (2023)
- [Ego4D: Around the World in 3,000 Hours of Egocentric Video](https://mlsanity.com/paper/2110.07058/) (2021)
- [The "something something" video database for learning and evaluating visual common sense](https://mlsanity.com/paper/1706.04261/) (2017)
- [Scaling Egocentric Vision: The EPIC-KITCHENS Dataset](https://mlsanity.com/paper/1804.02748/) (2018)
- [AVA: A Video Dataset of Spatio-temporally Localized Atomic Visual Actions](https://mlsanity.com/paper/1705.08421/) (2017)
- [Masked Autoencoders As Spatiotemporal Learners](https://mlsanity.com/paper/2205.09113/) (2022)
- [Partial success in closing the gap between human and machine vision](https://mlsanity.com/paper/2106.07411/) (2021)
- [A Short Note on the Kinetics-700-2020 Human Action Dataset](https://mlsanity.com/paper/2010.10864/) (2020)
- [Learning high-level visual representations from a child's perspective without strong inductive biases](https://mlsanity.com/paper/2305.15372/) (2023)
- [Self-supervised learning of video representations from a child's perspective](https://mlsanity.com/paper/2402.00300/) (2024)
- [Scaling may be all you need for achieving human-level object recognition capacity with human-like visual experience](https://mlsanity.com/paper/2308.03712/) (2023)

## Built on by (most cited first, 2 of 2)

- [Human Gaze Boosts Object-Centered Representation Learning](https://mlsanity.com/paper/2501.02966/) (2025)
- [Temporal Slowness in Central Vision Drives Semantic Object Learning](https://mlsanity.com/paper/2602.04462/) (2026)

## Abstract (arXiv)

We introduce Human-like Video Models (HVM-1), large-scale video models pretrained with nearly 5000 hours of curated human-like video data (mostly egocentric, temporally extended, continuous video recordings), using the spatiotemporal masked autoencoder (ST-MAE) algorithm. We release two 633M parameter models trained at spatial resolutions of 224x224 and 448x448 pixels. We evaluate the performance of these models in downstream few-shot video and image recognition tasks and compare them against a model pretrained with 1330 hours of short action-oriented video clips from YouTube (Kinetics-700). HVM-1 models perform competitively against the Kinetics-700 pretrained model in downstream evaluations despite substantial qualitative differences between the spatiotemporal characteristics of the corresponding pretraining datasets. HVM-1 models also learn more accurate and more robust object representations compared to models pretrained with the image-based MAE algorithm on the same data, demonstrating the potential benefits of learning to predict temporal regularities in natural videos for learning better object representations.