mlsanityData to 14 Oct 2026

Domain Generalization through Audio-Visual Relative Norm Alignment in First Person Action Recognition

arXiv 2110.10101 ↗First posted 19 Oct 2021cs.CVcited by 35

Not on the map. Why?

Abstract (arXiv)

First person action recognition is becoming an increasingly researched area thanks to the rising popularity of wearable cameras. This is bringing to light cross-domain issues that are yet to be addressed in this context. Indeed, the information extracted from learned representations suffers from an intrinsic "environmental bias". This strongly affects the ability to generalize to unseen scenarios, limiting the application of current methods to real settings where labeled data are not available during training. In this work, we introduce the first domain generalization approach for egocentric activity recognition, by proposing a new audio-visual loss, called Relative Norm Alignment loss. It re-balances the contributions from the two modalities during training, over different domains, by aligning their feature norm representations. Our approach leads to strong results in domain generalization on both EPIC-Kitchens-55 and EPIC-Kitchens-100, as demonstrated by extensive experiments, and can be extended to work also on domain adaptation settings with competitive results.

Key facts. Cited by 35 papers in the map (data through 14 Oct 2026). Builds on 70 of its 93 references that are in the map. First posted on arXiv 19 Oct 2021.

Builds on

70 of 93 references.

References · 70

  1. 2017Attention Is All You Needcited by 60,361
  2. 2015Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shiftcited by 7,005
  3. 2017Squeeze-and-Excitation Networkscited by 4,789
  4. 2017Quo Vadis, Action Recognition? A New Model and the Kinetics Datasetcited by 3,270
  5. 2016Making the V in VQA Matter: Elevating the Role of Image Understanding in Visual Question Answeringcited by 2,516
  6. 2017Non-local Neural Networkscited by 2,293
  7. 2014Unsupervised Domain Adaptation by Backpropagationcited by 1,917
  8. 2018SlowFast Networks for Video Recognitioncited by 1,867
  9. 2017Multi-Task Learning Using Uncertainty to Weigh Losses for Scene Geometry and Semanticscited by 1,358
  10. 2015Learning Transferable Features with Deep Adaptation Networkscited by 1,329
  11. 2014Two-Stream Convolutional Networks for Action Recognition in Videoscited by 1,068
  12. 2016Temporal Segment Networks: Towards Good Practices for Deep Action Recognitioncited by 902
38 more
  1. 2018TSM: Temporal Shift Module for Efficient Video Understandingcited by 856
  2. 2018Scaling Egocentric Vision: The EPIC-KITCHENS Datasetcited by 795
  3. 2017Maximum Classifier Discrepancy for Unsupervised Domain Adaptationcited by 683
  4. 2017Look, Listen and Learncited by 458
  5. 2018Generalizing to Unseen Domains via Adversarial Data Augmentationcited by 451
  6. 2019Domain Generalization by Solving Jigsaw Puzzlescited by 450
  7. 2019Domain Generalization via Model-Agnostic Learning of Semantic Featurescited by 407
  8. 2019What Makes Training Multi-Modal Classification Networks Hard?cited by 344
  9. 2018Audio-Visual Scene Analysis with Self-Supervised Multisensory Featurescited by 335
  10. 2017Temporal Relational Reasoning in Videoscited by 324
  11. 2018Hypothesis Only Baselines in Natural Language Inferencecited by 295
  12. 2016Revisiting Batch Normalization For Practical Domain Adaptationcited by 288
  13. 2019Self-Supervised Learning by Cross-Modal Audio-Video Clusteringcited by 267
  14. 2018The Sound of Pixelscited by 259
  15. 2017Objects that Soundcited by 256
  16. 2018Cooperative Learning of Audio and Video Models from Self-Supervised Synchronizationcited by 249
  17. 2018Long-Term Feature Banks for Detailed Video Understandingcited by 239
  18. 2017NormFace: L2 Hypersphere Embedding for Face Verificationcited by 237
  19. 2019EPIC-Fusion: Audio-Visual Temporal Binding for Egocentric Action Recognitioncited by 226
  20. 2017Improved Texture Networks: Maximizing Quality and Diversity in Feed-forward Stylization and Texture Synthesiscited by 215
  21. 2019Domain-Specific Batch Normalization for Unsupervised Domain Adaptationcited by 197
  22. 2020Audio-Visual Instance Discrimination with Cross-Modal Agreementcited by 195
  23. 2018Learning to Localize Sound Source in Visual Scenescited by 189
  24. 2020Self-Supervised Learning of Audio-Visual Objects from Videocited by 188
  25. 2019Listen to Look: Action Recognition by Previewing Audiocited by 163
  26. 2020Unified Multisensory Perception: Weakly-Supervised Audio-Visual Video Parsingcited by 146
  27. 2019Large-scale weakly-supervised pre-training for video action recognitioncited by 130
  28. 2019Audio-Visual Scene-Aware Dialogcited by 123
  29. 2020Rolling-Unrolling LSTMs for Action Anticipation from First-Person Videocited by 120
  30. 2020Multi-Modal Domain Adaptation for Fine-Grained Action Recognitioncited by 115
  31. 2018Rethinking the Smaller-Norm-Less-Informative Assumption in Channel Pruning of Convolution Layerscited by 112
  32. 2019Temporal Attentive Alignment for Large-Scale Video Domain Adaptationcited by 100
  33. 2017L2-constrained Softmax Loss for Discriminative Face Verificationcited by 96
  34. 2020Learning Representations from Audio-Visual Spatial Alignmentcited by 95
  35. 2019Cluster Alignment with a Teacher for Unsupervised Domain Adaptationcited by 88
  36. 2017Optical Flow Guided Feature: A Fast and Robust Motion Representation for Video Action Recognitioncited by 82
  37. 2021Robust Audio-Visual Instance Discriminationcited by 80
  38. 2018Batch-Instance Normalization for Adaptively Style-Invariant Neural Networkscited by 78

50 most cited of 70 All 70 (JSON)

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2021: 1; 2022: 6; 2023: 5; 2024: 8; 2025: 9; 2026: 6.

35 papers.

Sort:

  1. 2021E2(GO)MOTION: Motion Augmented Event Stream for Egocentric Action Recognitioncited by 50
  2. 2023SimMMDG: A Simple and Effective Framework for Multi-modal Domain Generalizationcited by 48
  3. 2022Learning in Audio-visual Context: A Review, Analysis, and New Perspectivecited by 44
  4. 2024MultiOOD: Scaling Out-of-Distribution Detection for Multiple Modalitiescited by 28
  5. 2023What can a cook in Italy teach a mechanic in India? Action Recognition Generalisation Over Scenarios and Locationscited by 25
  6. 2025Advances in Multimodal Adaptation and Generalization: From Traditional Approaches to Foundation Modelscited by 23
  7. 2023Diversifying Spatial-Temporal Perception for Video Domain Generalizationcited by 19
  8. 2024Towards Multimodal Open-Set Domain Generalization and Adaptation through Self-supervisioncited by 16
  9. 2024XOV-Action: Towards Generalizable Open-Vocabulary Action Recognitioncited by 15
  10. 2022Video Unsupervised Domain Adaptation with Deep Learning: A Comprehensive Surveycited by 12
  11. 2025Towards Robust Multimodal Open-set Test-time Adaptation via Adaptive Entropy-aware Optimizationcited by 12
  12. 2023Multimodal Imbalance-Aware Gradient Modulation for Weakly-supervised Audio-Visual Video Parsingcited by 11
23 more
  1. 2024Detached and Interactive Multimodal Learningcited by 11
  2. 2025Bridging Domain Generalization to Multimodal Domain Generalization via Unified Representationscited by 9
  3. 2024Human-Centric Transformer for Domain Adaptive Action Recognitioncited by 8
  4. 2022Towards Optimization and Model Selection for Domain Generalization: A Mixup-guided Solutioncited by 6
  5. 2025Modality-Balanced Collaborative Distillation for Multi-Modal Domain Generalizationcited by 6
  6. 2023Audio-Visual Glance Network for Efficient Video Recognitioncited by 4
  7. 2025Open-set Cross Modal Generalization via Multimodal Unified Representationcited by 4
  8. 2024Attend-Fusion: Efficient Audio-Visual Fusion for Video Classificationcited by 3
  9. 2025Exploring Audio Cues for Enhanced Test-Time Video Model Adaptationcited by 3
  10. 2024Integrating Audio Narrations to Strengthen Domain Generalization in Multimodal First-Person Action Recognitioncited by 2
  11. 2025Consistent and Invariant Generalization Learning for Short-video Misinformation Detectioncited by 2
  12. 2022Bringing Online Egocentric Action Recognition into the wildcited by 1
  13. 2025Exploring Scale Shift in Crowd Localization under the Context of Domain Generalizationcited by 1
  14. 2026Balancing Multimodal Domain Generalization via Gradient Modulation and Projectioncited by 1
  15. 2022PoliTO-IIT-CINI Submission to the EPIC-KITCHENS-100 Unsupervised Domain Adaptation Challenge for Action Recognitioncited by 0
  16. 2022A Survey on Human Action Recognitioncited by 0
  17. 2024Adversarially Masked Video Consistency for Unsupervised Domain Adaptationcited by 0
  18. 2025Domain Generalization for Improved Human Activity Recognition in Office Space Videos Using Adaptive Pre-processingcited by 0
  19. 2026MER-DG: Modality-Entropy Regularization for Multimodal Domain Generalizationcited by 0
  20. 2026Are We Making Progress in Multimodal Domain Generalization? A Comprehensive Benchmark Studycited by 0
  21. 2026Leveraging Self-Paced Curriculum Learning for Enhanced Modality Balance in Multimodal Conversational Emotion Recognitioncited by 0
  22. 2026MMDG-Bench: A Benchmark for Multimodal Domain Generalizationcited by 0
  23. 2026Multimodal Domain Generalization for Depression Detection: An Attention-Based BiLSTM Network with Domain-Adversarial Trainingcited by 0