# Semi-supervised 3D Video Information Retrieval with Deep Neural Network and Bi-directional Dynamic-time Warping Algorithm

arXiv: https://arxiv.org/abs/2309.01063 · HTML: https://mlsanity.com/paper/2309.01063/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 11 of its 35 references that are in the map.
- First posted on arXiv 3 Sep 2023.

## Builds on (most cited first, 11 of 11)

- [ImageNet Large Scale Visual Recognition Challenge](https://mlsanity.com/paper/1409.0575/) (2014)
- [Learning Phrase Representations using RNN Encoder-Decoder for Statistical Machine Translation](https://mlsanity.com/paper/1406.1078/) (2014)
- [Convolutional LSTM Network: A Machine Learning Approach for Precipitation Nowcasting](https://mlsanity.com/paper/1506.04214/) (2015)
- [Unsupervised Learning of Video Representations using LSTMs](https://mlsanity.com/paper/1502.04681/) (2015)
- [Video Action Transformer Network](https://mlsanity.com/paper/1812.02707/) (2018)
- [UnFlow: Unsupervised Learning of Optical Flow with a Bidirectional Census Loss](https://mlsanity.com/paper/1711.07837/) (2017)
- [Recycle-GAN: Unsupervised Video Retargeting](https://mlsanity.com/paper/1808.05174/) (2018)
- [Very Deep Convolutional Networks for End-to-End Speech Recognition](https://mlsanity.com/paper/1610.03022/) (2016)
- [WarpNet: Weakly Supervised Matching for Single-view Reconstruction](https://mlsanity.com/paper/1604.05592/) (2016)
- [Residual LSTM: Design of a Deep Recurrent Architecture for Distant Speech Recognition](https://mlsanity.com/paper/1701.03360/) (2017)
- [3D-CSL: self-supervised 3D context similarity learning for Near-Duplicate Video Retrieval](https://mlsanity.com/paper/2211.05352/) (2022)

## Abstract (arXiv)

This paper presents a novel semi-supervised deep learning algorithm for retrieving similar 2D and 3D videos based on visual content. The proposed approach combines the power of deep convolutional and recurrent neural networks with dynamic time warping as a similarity measure. The proposed algorithm is designed to handle large video datasets and retrieve the most related videos to a given inquiry video clip based on its graphical frames and contents. We split both the candidate and the inquiry videos into a sequence of clips and convert each clip to a representation vector using an autoencoder-backed deep neural network. We then calculate a similarity measure between the sequences of embedding vectors using a bi-directional dynamic time-warping method. This approach is tested on multiple public datasets, including CCWEBVIDEO, Youtube-8m, S3DIS, and Synthia, and showed good results compared to state-of-the-art. The algorithm effectively solves video retrieval tasks and outperforms the benchmarked state-of-the-art deep learning model.