# What can a cook in Italy teach a mechanic in India? Action Recognition Generalisation Over Scenarios and Locations

arXiv: https://arxiv.org/abs/2306.08713 · HTML: https://mlsanity.com/paper/2306.08713/

## Key facts

- Cited by 25 papers in the map (data through 14 Oct 2026).
- Builds on 50 of its 67 references that are in the map.
- First posted on arXiv 14 Jun 2023.

## Builds on (most cited first, 30 of 50)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Learning Transferable Visual Models From Natural Language Supervision](https://mlsanity.com/paper/2103.00020/) (2021)
- [Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks](https://mlsanity.com/paper/1908.10084/) (2019)
- [Representation Learning with Contrastive Predictive Coding](https://mlsanity.com/paper/1807.03748/) (2018)
- [Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift](https://mlsanity.com/paper/1502.03167/) (2015)
- [mixup: Beyond Empirical Risk Minimization](https://mlsanity.com/paper/1710.09412/) (2017)
- [Quo Vadis, Action Recognition? A New Model and the Kinetics Dataset](https://mlsanity.com/paper/1705.07750/) (2017)
- [Domain-Adversarial Training of Neural Networks](https://mlsanity.com/paper/1505.07818/) (2015)
- [SlowFast Networks for Video Recognition](https://mlsanity.com/paper/1812.03982/) (2018)
- [Ego4D: Around the World in 3,000 Hours of Egocentric Video](https://mlsanity.com/paper/2110.07058/) (2021)
- [MiniLM: Deep Self-Attention Distillation for Task-Agnostic Compression of Pre-Trained Transformers](https://mlsanity.com/paper/2002.10957/) (2020)
- [Moment Matching for Multi-Source Domain Adaptation](https://mlsanity.com/paper/1812.01754/) (2018)
- [Deep CORAL: Correlation Alignment for Deep Domain Adaptation](https://mlsanity.com/paper/1607.01719/) (2016)
- [Deep Hashing Network for Unsupervised Domain Adaptation](https://mlsanity.com/paper/1706.07522/) (2017)
- [In Search of Lost Domain Generalization](https://mlsanity.com/paper/2007.01434/) (2020)
- [Deeper, Broader and Artier Domain Generalization](https://mlsanity.com/paper/1710.03077/) (2017)
- [MPNet: Masked and Permuted Pre-training for Language Understanding](https://mlsanity.com/paper/2004.09297/) (2020)
- [Scaling Egocentric Vision: The EPIC-KITCHENS Dataset](https://mlsanity.com/paper/1804.02748/) (2018)
- [Learning to Generalize: Meta-Learning for Domain Generalization](https://mlsanity.com/paper/1710.03463/) (2017)
- [Recognition in Terra Incognita](https://mlsanity.com/paper/1807.04975/) (2018)
- [Rescaling Egocentric Vision](https://mlsanity.com/paper/2006.13256/) (2020)
- [Generalizing to Unseen Domains via Adversarial Data Augmentation](https://mlsanity.com/paper/1805.12018/) (2018)
- [Domain Generalization by Solving Jigsaw Puzzles](https://mlsanity.com/paper/1903.06864/) (2019)
- [Masked Autoencoders As Spatiotemporal Learners](https://mlsanity.com/paper/2205.09113/) (2022)
- [Domain Generalization via Model-Agnostic Learning of Semantic Features](https://mlsanity.com/paper/1910.13580/) (2019)
- [Test-Time Prompt Tuning for Zero-Shot Generalization in Vision-Language Models](https://mlsanity.com/paper/2209.07511/) (2022)
- [Temporal Relational Reasoning in Videos](https://mlsanity.com/paper/1711.08496/) (2017)
- [Adversarial Domain Adaptation with Domain Mixup](https://mlsanity.com/paper/1912.01805/) (2019)
- [Episodic Training for Domain Generalization](https://mlsanity.com/paper/1902.00113/) (2019)

## Built on by (most cited first, 25 of 25)

- [An Outlook into the Future of Egocentric Vision](https://mlsanity.com/paper/2308.07123/) (2023)
- [EgoVid-5M: A Large-Scale Video-Action Dataset for Egocentric Video Generation](https://mlsanity.com/paper/2411.08380/) (2024)
- [Challenges and Trends in Egocentric Vision: A Survey](https://mlsanity.com/paper/2503.15275/) (2025)
- [The Invisible EgoHand: 3D Hand Forecasting through EgoBody Pose Estimation](https://mlsanity.com/paper/2504.08654/) (2025)
- [XOV-Action: Towards Generalizable Open-Vocabulary Action Recognition](https://mlsanity.com/paper/2403.01560/) (2024)
- [Multimodal Cross-Domain Few-Shot Learning for Egocentric Action Recognition](https://mlsanity.com/paper/2405.19917/) (2024)
- [AFF-ttention! Affordances and Attention models for Short-Term Object Interaction Anticipation](https://mlsanity.com/paper/2406.01194/) (2024)
- [A Survey on Deep Learning Techniques for Action Anticipation](https://mlsanity.com/paper/2309.17257/) (2023)
- [Do Egocentric Video-Language Models Truly Understand Hand-Object Interactions?](https://mlsanity.com/paper/2405.17719/) (2024)
- [Are Synthetic Data Useful for Egocentric Hand-Object Interaction Detection?](https://mlsanity.com/paper/2312.02672/) (2023)
- [A Backpack Full of Skills: Egocentric Video Understanding with Diverse Task Perspectives](https://mlsanity.com/paper/2403.03037/) (2024)
- [What does CLIP know about peeling a banana?](https://mlsanity.com/paper/2404.12015/) (2024)
- [Human-Centric Transformer for Domain Adaptive Action Recognition](https://mlsanity.com/paper/2407.10860/) (2024)
- [Egocentric zone-aware action recognition across environments](https://mlsanity.com/paper/2409.14205/) (2024)
- [Hier-EgoPack: Hierarchical Egocentric Video Understanding with Diverse Task Perspectives](https://mlsanity.com/paper/2502.02487/) (2025)
- [EgoDTM: Towards 3D-Aware Egocentric Video-Language Pretraining](https://mlsanity.com/paper/2503.15470/) (2025)
- [Tell, Don't Show!: Language Guidance Eases Transfer Across Domains in Images and Videos](https://mlsanity.com/paper/2403.05535/) (2024)
- [Integrating Audio Narrations to Strengthen Domain Generalization in Multimodal First-Person Action Recognition](https://mlsanity.com/paper/2409.09611/) (2024)
- [Domain Generalization using Action Sequences for Egocentric Action Recognition](https://mlsanity.com/paper/2506.17685/) (2025)
- [Integrating Affordances and Attention models for Short-Term Object Interaction Anticipation](https://mlsanity.com/paper/2602.14837/) (2026)
- [VUDG: A Dataset for Video Understanding Domain Generalization](https://mlsanity.com/paper/2505.24346/) (2025)
- [EgoEMS: A High-Fidelity Multimodal Egocentric Dataset for Cognitive Assistance in Emergency Medical Services](https://mlsanity.com/paper/2511.09894/) (2025)
- [Ego4OOD: Rethinking Egocentric Video Domain Generalization via Covariate Shift Scoring](https://mlsanity.com/paper/2601.17056/) (2026)
- [SAVA-X: Ego-to-Exo Imitation Error Detection via Scene-Adaptive View Alignment and Bidirectional Cross View Fusion](https://mlsanity.com/paper/2603.12764/) (2026)
- [Leveraging Synthetic Data for Enhancing Egocentric Hand-Object Interaction Detection](https://mlsanity.com/paper/2603.29733/) (2026)

## Abstract (arXiv)

We propose and address a new generalisation problem: can a model trained for action recognition successfully classify actions when they are performed within a previously unseen scenario and in a previously unseen location? To answer this question, we introduce the Action Recognition Generalisation Over scenarios and locations dataset (ARGO1M), which contains 1.1M video clips from the large-scale Ego4D dataset, across 10 scenarios and 13 locations. We demonstrate recognition models struggle to generalise over 10 proposed test splits, each of an unseen scenario in an unseen location. We thus propose CIR, a method to represent each video as a Cross-Instance Reconstruction of videos from other domains. Reconstructions are paired with text narrations to guide the learning of a domain generalisable representation. We provide extensive analysis and ablations on ARGO1M that show CIR outperforms prior domain generalisation works on all test splits. Code and data: https://chiaraplizz.github.io/what-can-a-cook/.