Domain Generalization through Audio-Visual Relative Norm Alignment in First Person Action Recognition
Not on the map. Why?
Abstract (arXiv)
First person action recognition is becoming an increasingly researched area thanks to the rising popularity of wearable cameras. This is bringing to light cross-domain issues that are yet to be addressed in this context. Indeed, the information extracted from learned representations suffers from an intrinsic "environmental bias". This strongly affects the ability to generalize to unseen scenarios, limiting the application of current methods to real settings where labeled data are not available during training. In this work, we introduce the first domain generalization approach for egocentric activity recognition, by proposing a new audio-visual loss, called Relative Norm Alignment loss. It re-balances the contributions from the two modalities during training, over different domains, by aligning their feature norm representations. Our approach leads to strong results in domain generalization on both EPIC-Kitchens-55 and EPIC-Kitchens-100, as demonstrated by extensive experiments, and can be extended to work also on domain adaptation settings with competitive results.
Key facts. Cited by 35 papers in the map (data through 14 Oct 2026). Builds on 70 of its 93 references that are in the map. First posted on arXiv 19 Oct 2021.
Builds on
70 of 93 references.
References · 70
- 2017Attention Is All You Needcited by 60,361
- 2015Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shiftcited by 7,005
- 2017Squeeze-and-Excitation Networkscited by 4,789
- 2017Quo Vadis, Action Recognition? A New Model and the Kinetics Datasetcited by 3,270
- 2016Making the V in VQA Matter: Elevating the Role of Image Understanding in Visual Question Answeringcited by 2,516
- 2017Non-local Neural Networkscited by 2,293
- 2014Unsupervised Domain Adaptation by Backpropagationcited by 1,917
- 2018SlowFast Networks for Video Recognitioncited by 1,867
- 2017Multi-Task Learning Using Uncertainty to Weigh Losses for Scene Geometry and Semanticscited by 1,358
- 2015Learning Transferable Features with Deep Adaptation Networkscited by 1,329
- 2014Two-Stream Convolutional Networks for Action Recognition in Videoscited by 1,068
- 2016Temporal Segment Networks: Towards Good Practices for Deep Action Recognitioncited by 902
38 more
- 2018TSM: Temporal Shift Module for Efficient Video Understandingcited by 856
- 2018Scaling Egocentric Vision: The EPIC-KITCHENS Datasetcited by 795
- 2017Maximum Classifier Discrepancy for Unsupervised Domain Adaptationcited by 683
- 2017Look, Listen and Learncited by 458
- 2018Generalizing to Unseen Domains via Adversarial Data Augmentationcited by 451
- 2019Domain Generalization by Solving Jigsaw Puzzlescited by 450
- 2019Domain Generalization via Model-Agnostic Learning of Semantic Featurescited by 407
- 2019What Makes Training Multi-Modal Classification Networks Hard?cited by 344
- 2018Audio-Visual Scene Analysis with Self-Supervised Multisensory Featurescited by 335
- 2017Temporal Relational Reasoning in Videoscited by 324
- 2018Hypothesis Only Baselines in Natural Language Inferencecited by 295
- 2016Revisiting Batch Normalization For Practical Domain Adaptationcited by 288
- 2019Self-Supervised Learning by Cross-Modal Audio-Video Clusteringcited by 267
- 2018The Sound of Pixelscited by 259
- 2017Objects that Soundcited by 256
- 2018Cooperative Learning of Audio and Video Models from Self-Supervised Synchronizationcited by 249
- 2018Long-Term Feature Banks for Detailed Video Understandingcited by 239
- 2017NormFace: L2 Hypersphere Embedding for Face Verificationcited by 237
- 2019EPIC-Fusion: Audio-Visual Temporal Binding for Egocentric Action Recognitioncited by 226
- 2017Improved Texture Networks: Maximizing Quality and Diversity in Feed-forward Stylization and Texture Synthesiscited by 215
- 2019Domain-Specific Batch Normalization for Unsupervised Domain Adaptationcited by 197
- 2020Audio-Visual Instance Discrimination with Cross-Modal Agreementcited by 195
- 2018Learning to Localize Sound Source in Visual Scenescited by 189
- 2020Self-Supervised Learning of Audio-Visual Objects from Videocited by 188
- 2019Listen to Look: Action Recognition by Previewing Audiocited by 163
- 2020Unified Multisensory Perception: Weakly-Supervised Audio-Visual Video Parsingcited by 146
- 2019Large-scale weakly-supervised pre-training for video action recognitioncited by 130
- 2019Audio-Visual Scene-Aware Dialogcited by 123
- 2020Rolling-Unrolling LSTMs for Action Anticipation from First-Person Videocited by 120
- 2020Multi-Modal Domain Adaptation for Fine-Grained Action Recognitioncited by 115
- 2018Rethinking the Smaller-Norm-Less-Informative Assumption in Channel Pruning of Convolution Layerscited by 112
- 2019Temporal Attentive Alignment for Large-Scale Video Domain Adaptationcited by 100
- 2017L2-constrained Softmax Loss for Discriminative Face Verificationcited by 96
- 2020Learning Representations from Audio-Visual Spatial Alignmentcited by 95
- 2019Cluster Alignment with a Teacher for Unsupervised Domain Adaptationcited by 88
- 2017Optical Flow Guided Feature: A Fast and Robust Motion Representation for Video Action Recognitioncited by 82
- 2021Robust Audio-Visual Instance Discriminationcited by 80
- 2018Batch-Instance Normalization for Adaptively Style-Invariant Neural Networkscited by 78
50 most cited of 70 All 70 (JSON)
Built on by
Citing papers per year: 2021: 1; 2022: 6; 2023: 5; 2024: 8; 2025: 9; 2026: 6.
35 papers.
Sort:
- 2021E2(GO)MOTION: Motion Augmented Event Stream for Egocentric Action Recognitioncited by 50
- 2023SimMMDG: A Simple and Effective Framework for Multi-modal Domain Generalizationcited by 48
- 2022Learning in Audio-visual Context: A Review, Analysis, and New Perspectivecited by 44
- 2024MultiOOD: Scaling Out-of-Distribution Detection for Multiple Modalitiescited by 28
- 2023What can a cook in Italy teach a mechanic in India? Action Recognition Generalisation Over Scenarios and Locationscited by 25
- 2025Advances in Multimodal Adaptation and Generalization: From Traditional Approaches to Foundation Modelscited by 23
- 2023Diversifying Spatial-Temporal Perception for Video Domain Generalizationcited by 19
- 2024Towards Multimodal Open-Set Domain Generalization and Adaptation through Self-supervisioncited by 16
- 2024XOV-Action: Towards Generalizable Open-Vocabulary Action Recognitioncited by 15
- 2022Video Unsupervised Domain Adaptation with Deep Learning: A Comprehensive Surveycited by 12
- 2025Towards Robust Multimodal Open-set Test-time Adaptation via Adaptive Entropy-aware Optimizationcited by 12
- 2023Multimodal Imbalance-Aware Gradient Modulation for Weakly-supervised Audio-Visual Video Parsingcited by 11
23 more
- 2024Detached and Interactive Multimodal Learningcited by 11
- 2025Bridging Domain Generalization to Multimodal Domain Generalization via Unified Representationscited by 9
- 2024Human-Centric Transformer for Domain Adaptive Action Recognitioncited by 8
- 2022Towards Optimization and Model Selection for Domain Generalization: A Mixup-guided Solutioncited by 6
- 2025Modality-Balanced Collaborative Distillation for Multi-Modal Domain Generalizationcited by 6
- 2023Audio-Visual Glance Network for Efficient Video Recognitioncited by 4
- 2025Open-set Cross Modal Generalization via Multimodal Unified Representationcited by 4
- 2024Attend-Fusion: Efficient Audio-Visual Fusion for Video Classificationcited by 3
- 2025Exploring Audio Cues for Enhanced Test-Time Video Model Adaptationcited by 3
- 2024Integrating Audio Narrations to Strengthen Domain Generalization in Multimodal First-Person Action Recognitioncited by 2
- 2025Consistent and Invariant Generalization Learning for Short-video Misinformation Detectioncited by 2
- 2022Bringing Online Egocentric Action Recognition into the wildcited by 1
- 2025Exploring Scale Shift in Crowd Localization under the Context of Domain Generalizationcited by 1
- 2026Balancing Multimodal Domain Generalization via Gradient Modulation and Projectioncited by 1
- 2022PoliTO-IIT-CINI Submission to the EPIC-KITCHENS-100 Unsupervised Domain Adaptation Challenge for Action Recognitioncited by 0
- 2022A Survey on Human Action Recognitioncited by 0
- 2024Adversarially Masked Video Consistency for Unsupervised Domain Adaptationcited by 0
- 2025Domain Generalization for Improved Human Activity Recognition in Office Space Videos Using Adaptive Pre-processingcited by 0
- 2026MER-DG: Modality-Entropy Regularization for Multimodal Domain Generalizationcited by 0
- 2026Are We Making Progress in Multimodal Domain Generalization? A Comprehensive Benchmark Studycited by 0
- 2026Leveraging Self-Paced Curriculum Learning for Enhanced Modality Balance in Multimodal Conversational Emotion Recognitioncited by 0
- 2026MMDG-Bench: A Benchmark for Multimodal Domain Generalizationcited by 0
- 2026Multimodal Domain Generalization for Depression Detection: An Attention-Based BiLSTM Network with Domain-Adversarial Trainingcited by 0
- 2026Multimodal Domain Generalization for Depression Detection: An Attention-Based BiLSTM Network with Domain-Adversarial Training24 Jul 2026
- 2026MMDG-Bench: A Benchmark for Multimodal Domain Generalization30 May 2026
- 2026Leveraging Self-Paced Curriculum Learning for Enhanced Modality Balance in Multimodal Conversational Emotion Recognition20 May 2026
- 2026Are We Making Progress in Multimodal Domain Generalization? A Comprehensive Benchmark Study7 May 2026
- 2026MER-DG: Modality-Entropy Regularization for Multimodal Domain Generalization3 May 2026
- 2026Balancing Multimodal Domain Generalization via Gradient Modulation and Projection15 Mar 2026
- 2025Modality-Balanced Collaborative Distillation for Multi-Modal Domain Generalization25 Nov 2025
- 2025Exploring Scale Shift in Crowd Localization under the Context of Domain Generalization22 Oct 2025
- 2025Open-set Cross Modal Generalization via Multimodal Unified Representation20 Jul 2025
- 2025Consistent and Invariant Generalization Learning for Short-video Misinformation Detection5 Jul 2025
- 2025Bridging Domain Generalization to Multimodal Domain Generalization via Unified Representations4 Jul 2025
- 2025Exploring Audio Cues for Enhanced Test-Time Video Model Adaptation14 Jun 2025
23 more
- 2025Domain Generalization for Improved Human Activity Recognition in Office Space Videos Using Adaptive Pre-processing16 Mar 2025
- 2025Advances in Multimodal Adaptation and Generalization: From Traditional Approaches to Foundation Models30 Jan 2025
- 2025Towards Robust Multimodal Open-set Test-time Adaptation via Adaptive Entropy-aware Optimization23 Jan 2025
- 2024Integrating Audio Narrations to Strengthen Domain Generalization in Multimodal First-Person Action Recognition15 Sep 2024
- 2024Attend-Fusion: Efficient Audio-Visual Fusion for Video Classification26 Aug 2024
- 2024Detached and Interactive Multimodal Learning28 Jul 2024
- 2024Human-Centric Transformer for Domain Adaptive Action Recognition15 Jul 2024
- 2024Towards Multimodal Open-Set Domain Generalization and Adaptation through Self-supervision1 Jul 2024
- 2024MultiOOD: Scaling Out-of-Distribution Detection for Multiple Modalities27 May 2024
- 2024Adversarially Masked Video Consistency for Unsupervised Domain Adaptation24 Mar 2024
- 2024XOV-Action: Towards Generalizable Open-Vocabulary Action Recognition3 Mar 2024
- 2023SimMMDG: A Simple and Effective Framework for Multi-modal Domain Generalization30 Oct 2023
- 2023Diversifying Spatial-Temporal Perception for Video Domain Generalization27 Oct 2023
- 2023Audio-Visual Glance Network for Efficient Video Recognition18 Aug 2023
- 2023Multimodal Imbalance-Aware Gradient Modulation for Weakly-supervised Audio-Visual Video Parsing5 Jul 2023
- 2023What can a cook in Italy teach a mechanic in India? Action Recognition Generalisation Over Scenarios and Locations14 Jun 2023
- 2022A Survey on Human Action Recognition20 Dec 2022
- 2022Video Unsupervised Domain Adaptation with Deep Learning: A Comprehensive Survey17 Nov 2022
- 2022Bringing Online Egocentric Action Recognition into the wild6 Nov 2022
- 2022PoliTO-IIT-CINI Submission to the EPIC-KITCHENS-100 Unsupervised Domain Adaptation Challenge for Action Recognition9 Sep 2022
- 2022Towards Optimization and Model Selection for Domain Generalization: A Mixup-guided Solution1 Sep 2022
- 2022Learning in Audio-visual Context: A Review, Analysis, and New Perspective20 Aug 2022
- 2021E2(GO)MOTION: Motion Augmented Event Stream for Egocentric Action Recognition7 Dec 2021
- 2021E2(GO)MOTION: Motion Augmented Event Stream for Egocentric Action Recognitioncited by 50
- 2023SimMMDG: A Simple and Effective Framework for Multi-modal Domain Generalizationcited by 48
- 2022Learning in Audio-visual Context: A Review, Analysis, and New Perspectivecited by 44
- 2024MultiOOD: Scaling Out-of-Distribution Detection for Multiple Modalitiescited by 28
- 2023What can a cook in Italy teach a mechanic in India? Action Recognition Generalisation Over Scenarios and Locationscited by 25
- 2025Advances in Multimodal Adaptation and Generalization: From Traditional Approaches to Foundation Modelscited by 23
- 2023Diversifying Spatial-Temporal Perception for Video Domain Generalizationcited by 19
- 2024Towards Multimodal Open-Set Domain Generalization and Adaptation through Self-supervisioncited by 16
- 2024XOV-Action: Towards Generalizable Open-Vocabulary Action Recognitioncited by 15
- 2022Video Unsupervised Domain Adaptation with Deep Learning: A Comprehensive Surveycited by 12
- 2025Towards Robust Multimodal Open-set Test-time Adaptation via Adaptive Entropy-aware Optimizationcited by 12
- 2023Multimodal Imbalance-Aware Gradient Modulation for Weakly-supervised Audio-Visual Video Parsingcited by 11
12 most cited of 35 All 35 (JSON)
- 2026Multimodal Domain Generalization for Depression Detection: An Attention-Based BiLSTM Network with Domain-Adversarial Training24 Jul 2026
- 2026MMDG-Bench: A Benchmark for Multimodal Domain Generalization30 May 2026
- 2026Leveraging Self-Paced Curriculum Learning for Enhanced Modality Balance in Multimodal Conversational Emotion Recognition20 May 2026
- 2026Are We Making Progress in Multimodal Domain Generalization? A Comprehensive Benchmark Study7 May 2026
- 2026MER-DG: Modality-Entropy Regularization for Multimodal Domain Generalization3 May 2026
- 2026Balancing Multimodal Domain Generalization via Gradient Modulation and Projection15 Mar 2026
- 2025Modality-Balanced Collaborative Distillation for Multi-Modal Domain Generalization25 Nov 2025
- 2025Exploring Scale Shift in Crowd Localization under the Context of Domain Generalization22 Oct 2025
- 2025Open-set Cross Modal Generalization via Multimodal Unified Representation20 Jul 2025
- 2025Consistent and Invariant Generalization Learning for Short-video Misinformation Detection5 Jul 2025
- 2025Bridging Domain Generalization to Multimodal Domain Generalization via Unified Representations4 Jul 2025
- 2025Exploring Audio Cues for Enhanced Test-Time Video Model Adaptation14 Jun 2025
12 newest of 35 All 35 (JSON)