mlsanityData to 7 Oct 2026

M&M Mix: A Multimodal Multiview Transformer Ensemble

arXiv 2206.09852 ↗First posted 20 Jun 2022cs.CVcited by 19

Not on the map. Why?

Abstract (arXiv)

This report describes the approach behind our winning solution to the 2022 Epic-Kitchens Action Recognition Challenge. Our approach builds upon our recent work, Multiview Transformer for Video Recognition (MTV), and adapts it to multimodal inputs. Our final submission consists of an ensemble of Multimodal MTV (M&M) models varying backbone sizes and input modalities. Our approach achieved 52.8% Top-1 accuracy on the test set in action classes, which is 4.1% higher than last year's winning entry.

Key facts. Cited by 19 papers in the map (data through 7 Oct 2026). Builds on 28 of its 29 references that are in the map. First posted on arXiv 20 Jun 2022.

Builds on

28 of 29 references.

References · 28

  1. 2017Attention Is All You Needcited by 59,932
  2. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  3. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 29,915
  4. 2015Rethinking the Inception Architecture for Computer Visioncited by 5,451
  5. 2020Training data-efficient image transformers & distillation through attentioncited by 4,560
  6. 2017Quo Vadis, Action Recognition? A New Model and the Kinetics Datasetcited by 3,259
  7. 2019RandAugment: Practical automated data augmentation with a reduced search spacecited by 2,261
  8. 2017The Kinetics Human Action Video Datasetcited by 1,894
  9. 2018SlowFast Networks for Video Recognitioncited by 1,862
  10. 2021ViViT: A Video Vision Transformercited by 1,610
  11. 2019SpecAugment: A Simple Data Augmentation Method for Automatic Speech Recognitioncited by 1,355
  12. 2014Two-Stream Convolutional Networks for Action Recognition in Videoscited by 1,067
16 more
  1. 2015FlowNet: Learning Optical Flow with Convolutional Networkscited by 1,054
  2. 2016Deep Networks with Stochastic Depthcited by 973
  3. 2016CNN Architectures for Large-Scale Audio Classificationcited by 701
  4. 2021How to train your ViT? Data, Augmentation, and Regularization in Vision Transformerscited by 562
  5. 2020Rescaling Egocentric Visioncited by 529
  6. 2020Multi-modal Transformer for Video Retrievalcited by 395
  7. 2021Attention Bottlenecks for Multimodal Fusioncited by 361
  8. 2019EPIC-Fusion: Audio-Visual Temporal Binding for Egocentric Action Recognitioncited by 226
  9. 2018Moments in Time Dataset: one million videos for event understandingcited by 215
  10. 2022Omnivore: A Single Model for Many Visual Modalitiescited by 214
  11. 2022MeMViT: Memory-Augmented Multiscale Vision Transformer for Efficient Long-Term Video Recognitioncited by 189
  12. 2022Multiview Transformers for Video Recognitioncited by 148
  13. 2021MoViNets: Mobile Video Networks for Efficient Video Recognitioncited by 135
  14. 2021SCENIC: A JAX Library for Computer Vision Research and Beyondcited by 56
  15. 2021With a Little Help from my Temporal Context: Multimodal Egocentric Action Recognitioncited by 41
  16. 2020Learning Video Representations from Textual Web Supervisioncited by 39

Built on by

Cited by year

Citing papers per year: 2022: 3; 2023: 9; 2024: 3; 2025: 4.

19 papers.

Sort:

By area:

  1. 2023An Outlook into the Future of Egocentric Visioncited by 84
  2. 2023Epic-Sounds: A Large-scale Dataset of Actions That Soundcited by 47
  3. 2023Multimodal Distillation for Egocentric Action Recognitioncited by 28
  4. 2022Vision Transformers for Action Recognition: A Surveycited by 25
  5. 2023Training a Large Video Model on a Single Machine in a Daycited by 20
  6. 2024TIM: A Time Interval Machine for Audio-Visual Action Recognitioncited by 15
  7. 2023Cross-view Action Recognition Understanding From Exocentric to Egocentric Perspectivecited by 14
  8. 2024X-MIC: Cross-Modal Instance Conditioning for Egocentric Action Generalizationcited by 6
  9. 2023Optimizing ViViT Training: Time and Memory Reduction for Action Recognitioncited by 4
  10. 2023MOFO: MOtion FOcused Self-Supervision for Video Understandingcited by 3
  11. 2025LLaVAction: evaluating and training multi-modal large language models for action understandingcited by 3
  12. 2022Students taught by multimodal teachers are superior action recognizerscited by 2
7 more
  1. 2022Deep Architectures for Content Moderation and Movie Content Ratingcited by 2
  2. 2023Team AcieLee: Technical Report for EPIC-SOUNDS Audio-Based Interaction Recognition Challenge 2023cited by 0
  3. 2023IndGIC: Supervised Action Recognition under Low Illuminationcited by 0
  4. 2024Sensitive Image Classification by Vision Transformerscited by 0
  5. 2025CM3T: Framework for Efficient Multimodal Learning for Inhomogeneous Interaction Datasetscited by 0
  6. 2025Multimodal Knowledge Distillation for Egocentric Action Recognition Robust to Missing Modalitiescited by 0
  7. 2025Improving Keystep Recognition in Ego-Video via Dexterous Focuscited by 0