# Feature Re-Learning with Data Augmentation for Video Relevance Prediction

arXiv: https://arxiv.org/abs/2004.03815 · HTML: https://mlsanity.com/paper/2004.03815/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 8 Apr 2020.

## Built on by (most cited first, 5 of 5)

- [Coarse to Fine Multi-Resolution Temporal Convolutional Network](https://mlsanity.com/paper/2105.10859/) (2021)
- [A Feature-space Multimodal Data Augmentation Technique for Text-video Retrieval](https://mlsanity.com/paper/2208.02080/) (2022)
- [Fine-Grained Fashion Similarity Prediction by Attribute-Specific Embedding Learning](https://mlsanity.com/paper/2104.02429/) (2021)
- [TSMS-SAM2: Multi-scale Temporal Sampling Augmentation and Memory-Splitting Pruning for Promptable Video Object Segmentation and Tracking in Surgical Scenarios](https://mlsanity.com/paper/2508.05829/) (2025)
- [LAVA: Language Driven Scalable and Versatile Traffic Video Analytics](https://mlsanity.com/paper/2507.19821/) (2025)

## Abstract (arXiv)

Predicting the relevance between two given videos with respect to their visual content is a key component for content-based video recommendation and retrieval. Thanks to the increasing availability of pre-trained image and video convolutional neural network models, deep visual features are widely used for video content representation. However, as how two videos are relevant is task-dependent, such off-the-shelf features are not always optimal for all tasks. Moreover, due to varied concerns including copyright, privacy and security, one might have access to only pre-computed video features rather than original videos. We propose in this paper feature re-learning for improving video relevance prediction, with no need of revisiting the original video content. In particular, re-learning is realized by projecting a given deep feature into a new space by an affine transformation. We optimize the re-learning process by a novel negative-enhanced triplet ranking loss. In order to generate more training data, we propose a new data augmentation strategy which works directly on frame-level and video-level features. Extensive experiments in the context of the Hulu Content-based Video Relevance Prediction Challenge 2018 justify the effectiveness of the proposed method and its state-of-the-art performance for content-based video relevance prediction.