# CPFD: Confidence-aware Privileged Feature Distillation for Short Video Classification

arXiv: https://arxiv.org/abs/2410.03038 · HTML: https://mlsanity.com/paper/2410.03038/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- Placed in General knowledge distillation methods, one of 12 directions in Knowledge distillation.
- Builds on 16 of its 24 references that are in the map.
- First posted on arXiv 3 Oct 2024.

## Placed in

- Knowledge distillation / [General knowledge distillation methods](https://mlsanity.com/distill/general-knowledge-distillation-methods/)

## Builds on (most cited first, 16 of 16)

- [Distilling the Knowledge in a Neural Network](https://mlsanity.com/paper/1503.02531/) (2015)
- [BLIP: Bootstrapping Language-Image Pre-training for Unified Vision-Language Understanding and Generation](https://mlsanity.com/paper/2201.12086/) (2022)
- [Co-teaching: Robust Training of Deep Neural Networks with Extremely Noisy Labels](https://mlsanity.com/paper/1804.06872/) (2018)
- [SimVLM: Simple Visual Language Model Pretraining with Weak Supervision](https://mlsanity.com/paper/2108.10904/) (2021)
- [Combating noisy labels by agreement: A joint training method with co-regularization](https://mlsanity.com/paper/2003.02752/) (2020)
- [An Empirical Study of Training End-to-End Vision-and-Language Transformers](https://mlsanity.com/paper/2111.02387/) (2021)
- [Multi-Grained Vision Language Pre-Training: Aligning Texts with Visual Concepts](https://mlsanity.com/paper/2111.08276/) (2021)
- [Unifying distillation and privileged information](https://mlsanity.com/paper/1511.03643/) (2015)
- [Confidence-Aware Imitation Learning from Demonstrations with Varying Optimality](https://mlsanity.com/paper/2110.14754/) (2021)
- [HateMM: A Multi-Modal Dataset for Hate Video Classification](https://mlsanity.com/paper/2305.03915/) (2023)
- [FakeSV: A Multimodal Benchmark with Rich Social Context for Fake News Detection on Short Video Platforms](https://mlsanity.com/paper/2211.10973/) (2022)
- [Prime-Aware Adaptive Distillation](https://mlsanity.com/paper/2008.01458/) (2020)
- [Privileged Features Distillation at Taobao Recommendations](https://mlsanity.com/paper/1907.05171/) (2019)
- [Toward Understanding Privileged Features Distillation in Learning-to-Rank](https://mlsanity.com/paper/2209.08754/) (2022)
- [Calibration-compatible Listwise Distillation of Privileged Features for CTR Prediction](https://mlsanity.com/paper/2312.08727/) (2023)
- [Node Cardinality Estimation in the Internet of Things Using Privileged Feature Distillation](https://mlsanity.com/paper/2310.18664/) (2023)

## Built on by (most cited first, 4 of 4)

- [Embedding-based Retrieval in Multimodal Content Moderation](https://mlsanity.com/paper/2507.01066/) (2025)
- [UNIVID: Unified Vision-Language Model for Video Moderation](https://mlsanity.com/paper/2606.05748/) (2026)
- [PinSieve: Production Selective VLM Serving and a Governed Memory Flywheel for Enterprise Content-Quality Triage](https://mlsanity.com/paper/2608.24040/) (2026)
- [Do Input-Level Defenses Transfer to Observation-Level Attacks on VideoLLMs?](https://mlsanity.com/paper/2609.08331/) (2026)

## Abstract (arXiv)

Dense features, customized for different business scenarios, are essential in short video classification. However, their complexity, specific adaptation requirements, and high computational costs make them resource-intensive and less accessible during online inference. Consequently, these dense features are categorized as `Privileged Dense Features'.Meanwhile, end-to-end multi-modal models have shown promising results in numerous computer vision tasks. In industrial applications, prioritizing end-to-end multi-modal features, can enhance efficiency but often leads to the loss of valuable information from historical privileged dense features. To integrate both features while maintaining efficiency and manageable resource costs, we present Confidence-aware Privileged Feature Distillation (CPFD), which empowers features of an end-to-end multi-modal model by adaptively distilling privileged features during training. Unlike existing privileged feature distillation (PFD) methods, which apply uniform weights to all instances during distillation, potentially causing unstable performance across different business scenarios and a notable performance gap between teacher model (Dense Feature enhanced multimodal-model DF-X-VLM) and student model (multimodal-model only X-VLM), our CPFD leverages confidence scores derived from the teacher model to adaptively mitigate the performance variance with the student model. We conducted extensive offline experiments on five diverse tasks demonstrating that CPFD improves the video classification F1 score by 6.76% compared with end-to-end multimodal-model (X-VLM) and by 2.31% with vanilla PFD on-average. And it reduces the performance gap by 84.6% and achieves results comparable to teacher model DF-X-VLM. The effectiveness of CPFD is further substantiated by online experiments, and our framework has been deployed in production systems for over a dozen models.