# DMCL: Distillation Multiple Choice Learning for Multimodal Action Recognition

arXiv: https://arxiv.org/abs/1912.10982 · HTML: https://mlsanity.com/paper/1912.10982/

## Key facts

- Cited by 30 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 23 Dec 2019.

## Built on by (most cited first, 30 of 30)

- [A Comprehensive Survey on Source-free Domain Adaptation](https://mlsanity.com/paper/2302.11803/) (2023)
- [Towards Good Practices for Missing Modality Robust Action Recognition](https://mlsanity.com/paper/2211.13916/) (2022)
- [Learning an Augmented RGB Representation with Cross-Modal Knowledge Distillation for Action Detection](https://mlsanity.com/paper/2108.03619/) (2021)
- [A Review on Methods and Applications in Multimodal Deep Learning](https://mlsanity.com/paper/2202.09195/) (2022)
- [Robust Multimodal Learning with Missing Modalities via Parameter-Efficient Adaptation](https://mlsanity.com/paper/2310.03986/) (2023)
- [Multimodal Distillation for Egocentric Action Recognition](https://mlsanity.com/paper/2307.07483/) (2023)
- [DP-SSL: Towards Robust Semi-supervised Learning with A Few Labeled Samples](https://mlsanity.com/paper/2110.13740/) (2021)
- [Cross-Modal Knowledge Transfer Without Task-Relevant Source Data](https://mlsanity.com/paper/2209.04027/) (2022)
- [Dealing with Missing Modalities in the Visual Question Answer-Difference Prediction Task through Knowledge Distillation](https://mlsanity.com/paper/2104.05965/) (2021)
- [Resilient Multiple Choice Learning: A learned scoring scheme with application to audio scene analysis](https://mlsanity.com/paper/2311.01052/) (2023)
- [Annealed Multiple Choice Learning: Overcoming limitations of Winner-takes-all with annealing](https://mlsanity.com/paper/2407.15580/) (2024)
- [A Survey on Multimodal Wearable Sensor-based Human Action Recognition](https://mlsanity.com/paper/2404.15349/) (2024)
- [From Samples to Scenarios: A New Paradigm for Probabilistic Forecasting](https://mlsanity.com/paper/2509.19975/) (2025)
- [Modality Mixer for Multi-modal Action Recognition](https://mlsanity.com/paper/2208.11314/) (2022)
- [COMODO: Cross-Modal Video-to-IMU Distillation for Efficient Egocentric Human Activity Recognition](https://mlsanity.com/paper/2503.07259/) (2025)
- [Progressive Cross-modal Knowledge Distillation for Human Action Recognition](https://mlsanity.com/paper/2208.08090/) (2022)
- [MV2MAE: Multi-View Video Masked Autoencoders](https://mlsanity.com/paper/2401.15900/) (2024)
- [Winner-takes-all learners are geometry-aware conditional density estimators](https://mlsanity.com/paper/2406.04706/) (2024)
- [MATPAC++: Enhanced Masked Latent Prediction for Self-Supervised Audio Representation Learning](https://mlsanity.com/paper/2508.12709/) (2025)
- [Adaptive Pseudo-Label Refinement by Negative Ensemble Learning for Source-Free Unsupervised Domain Adaptation](https://mlsanity.com/paper/2103.15973/) (2021)
- [Source-Free Cross-Modal Knowledge Transfer by Unleashing the Potential of Task-Irrelevant Data](https://mlsanity.com/paper/2401.05014/) (2024)
- [Winner-takes-all for Multivariate Probabilistic Time Series Forecasting](https://mlsanity.com/paper/2506.05515/) (2025)
- [Pose-Aware Weakly-Supervised Action Segmentation](https://mlsanity.com/paper/2504.05700/) (2025)
- [Multiple Choice Learning of Low-Rank Adapters for Language Modeling](https://mlsanity.com/paper/2507.10419/) (2025)
- [Distillation of Human-Object Interaction Contexts for Action Recognition](https://mlsanity.com/paper/2112.09448/) (2021)
- [Multimodal Knowledge Distillation for Egocentric Action Recognition Robust to Missing Modalities](https://mlsanity.com/paper/2504.08578/) (2025)
- [View-aware Cross-modal Distillation for Multi-view Action Recognition](https://mlsanity.com/paper/2511.12870/) (2025)
- [Feature-Supervised Action Modality Transfer](https://mlsanity.com/paper/2108.03329/) (2021)
- [Modality Mixer Exploiting Complementary Information for Multi-modal Action Recognition](https://mlsanity.com/paper/2311.12344/) (2023)
- [An Attention-based Model for Robust Forecasting with Missing Modality](https://mlsanity.com/paper/2606.13970/) (2026)

## Abstract (arXiv)

In this work, we address the problem of learning an ensemble of specialist networks using multimodal data, while considering the realistic and challenging scenario of possible missing modalities at test time. Our goal is to leverage the complementary information of multiple modalities to the benefit of the ensemble and each individual network. We introduce a novel Distillation Multiple Choice Learning framework for multimodal data, where different modality networks learn in a cooperative setting from scratch, strengthening one another. The modality networks learned using our method achieve significantly higher accuracy than if trained separately, due to the guidance of other modalities. We evaluate this approach on three video action recognition benchmark datasets. We obtain state-of-the-art results in comparison to other approaches that work with missing modalities at test time.