# Cross-modal knowledge distillation for action recognition

arXiv: https://arxiv.org/abs/1910.04641 · HTML: https://mlsanity.com/paper/1910.04641/

## Key facts

- Cited by 45 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 10 Oct 2019.

## Built on by (most cited first, 30 of 45)

- [Learning Student-Friendly Teacher Networks for Knowledge Distillation](https://mlsanity.com/paper/2102.07650/) (2021)
- [EvDistill: Asynchronous Events to End-task Learning via Bidirectional Reconstruction-guided Cross-modal Knowledge Distillation](https://mlsanity.com/paper/2111.12341/) (2021)
- [The Modality Focusing Hypothesis: Towards Understanding Crossmodal Knowledge Distillation](https://mlsanity.com/paper/2206.06487/) (2022)
- [A Survey on Heterogeneous Federated Learning](https://mlsanity.com/paper/2210.04505/) (2022)
- [PartDistill: 3D Shape Part Segmentation by Vision-Language Model Distillation](https://mlsanity.com/paper/2312.04016/) (2023)
- [Learning from Temporal Gradient for Semi-supervised Action Recognition](https://mlsanity.com/paper/2111.13241/) (2021)
- [EventDance: Unsupervised Source-free Cross-modal Adaptation for Event-based Object Recognition](https://mlsanity.com/paper/2403.14082/) (2024)
- [Categories of Response-Based, Feature-Based, and Relation-Based Knowledge Distillation](https://mlsanity.com/paper/2306.10687/) (2023)
- [DnS: Distill-and-Select for Efficient and Accurate Video Indexing and Retrieval](https://mlsanity.com/paper/2106.13266/) (2021)
- [Cross-Modal Knowledge Transfer Without Task-Relevant Source Data](https://mlsanity.com/paper/2209.04027/) (2022)
- [CrossGLG: LLM Guides One-shot Skeleton-based 3D Action Recognition in a Cross-level Manner](https://mlsanity.com/paper/2403.10082/) (2024)
- [A Survey on Multimodal Wearable Sensor-based Human Action Recognition](https://mlsanity.com/paper/2404.15349/) (2024)
- [Cross-Modal Knowledge Distillation Method for Automatic Cued Speech Recognition](https://mlsanity.com/paper/2106.13686/) (2021)
- [Transferring Knowledge Distillation for Multilingual Social Event Detection](https://mlsanity.com/paper/2108.03084/) (2021)
- [Generative Model-based Feature Knowledge Distillation for Action Recognition](https://mlsanity.com/paper/2312.08644/) (2023)
- [Self-Adapting Large Visual-Language Models to Edge Devices across Visual Modalities](https://mlsanity.com/paper/2403.04908/) (2024)
- [Progressive Cross-modal Knowledge Distillation for Human Action Recognition](https://mlsanity.com/paper/2208.08090/) (2022)
- [Text-Derived Knowledge Helps Vision: A Simple Cross-modal Distillation for Video-based Action Anticipation](https://mlsanity.com/paper/2210.05991/) (2022)
- [MoMA: Momentum Contrastive Learning with Multi-head Attention-based Knowledge Distillation for Histopathology Image Analysis](https://mlsanity.com/paper/2308.16561/) (2023)
- [VideoAdviser: Video Knowledge Distillation for Multimodal Transfer Learning](https://mlsanity.com/paper/2309.15494/) (2023)
- [Segment Any Events via Weighted Adaptation of Pivotal Tokens](https://mlsanity.com/paper/2312.16222/) (2023)
- [Distilling Cross-Modal Knowledge via Feature Disentanglement](https://mlsanity.com/paper/2511.19887/) (2025)
- [Federated Action Recognition on Heterogeneous Embedded Devices](https://mlsanity.com/paper/2107.12147/) (2021)
- [Enabling All In-Edge Deep Learning: A Literature Review](https://mlsanity.com/paper/2204.03326/) (2022)
- [Audio Representation Learning by Distilling Video as Privileged Information](https://mlsanity.com/paper/2302.02845/) (2023)
- [Cross-sensor self-supervised training and alignment for remote sensing](https://mlsanity.com/paper/2405.09922/) (2024)
- [Enhancing Action Recognition from Low-Quality Skeleton Data via Part-Level Knowledge Distillation](https://mlsanity.com/paper/2404.18206/) (2024)
- [C3T: Cross-modal Transfer Through Time for Sensor-based Human Activity Recognition](https://mlsanity.com/paper/2407.16803/) (2024)
- [End-to-End Automatic Speech Recognition with Deep Mutual Learning](https://mlsanity.com/paper/2102.08154/) (2021)
- [Distillation of Human-Object Interaction Contexts for Action Recognition](https://mlsanity.com/paper/2112.09448/) (2021)

## Abstract (arXiv)

In this work, we address the problem how a network for action recognition that has been trained on a modality like RGB videos can be adapted to recognize actions for another modality like sequences of 3D human poses. To this end, we extract the knowledge of the trained teacher network for the source modality and transfer it to a small ensemble of student networks for the target modality. For the cross-modal knowledge distillation, we do not require any annotated data. Instead we use pairs of sequences of both modalities as supervision, which are straightforward to acquire. In contrast to previous works for knowledge distillation that use a KL-loss, we show that the cross-entropy loss together with mutual learning of a small ensemble of student networks performs better. In fact, the proposed approach for cross-modal knowledge distillation nearly achieves the accuracy of a student network trained with full supervision.