# A multimodal deep learning framework for scalable content based visual media retrieval

arXiv: https://arxiv.org/abs/2105.08665 · HTML: https://mlsanity.com/paper/2105.08665/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 19 of its 55 references that are in the map.
- First posted on arXiv 18 May 2021.

## Builds on (most cited first, 19 of 19)

- [MobileNets: Efficient Convolutional Neural Networks for Mobile Vision Applications](https://mlsanity.com/paper/1704.04861/) (2017)
- [Billion-scale similarity search with GPUs](https://mlsanity.com/paper/1702.08734/) (2017)
- [End-to-End Learning of Visual Representations from Uncurated Instructional Videos](https://mlsanity.com/paper/1912.06430/) (2019)
- [Long-term Recurrent Convolutional Networks for Visual Recognition and Description](https://mlsanity.com/paper/1411.4389/) (2014)
- [Beyond Short Snippets: Deep Networks for Video Classification](https://mlsanity.com/paper/1503.08909/) (2015)
- [Learning Fine-grained Image Similarity with Deep Ranking](https://mlsanity.com/paper/1404.4661/) (2014)
- [Deep Image Retrieval: Learning global representations for image search](https://mlsanity.com/paper/1604.01325/) (2016)
- [End-to-end Learning of Deep Visual Representations for Image Retrieval](https://mlsanity.com/paper/1610.07940/) (2016)
- [A Survey on Learning to Hash](https://mlsanity.com/paper/1606.00185/) (2016)
- [ECO: Efficient Convolutional Network for Online Video Understanding](https://mlsanity.com/paper/1804.09066/) (2018)
- [A Decade Survey of Content Based Image Retrieval using Deep Learning](https://mlsanity.com/paper/2012.00641/) (2020)
- [Aggregating Deep Convolutional Features for Image Retrieval](https://mlsanity.com/paper/1510.07493/) (2015)
- [Approximate Nearest Neighbor Search on High Dimensional Data --- Experiments, Analyses, and Improvement (v1.0)](https://mlsanity.com/paper/1610.02455/) (2016)
- [Learning from Video and Text via Large-Scale Discriminative Clustering](https://mlsanity.com/paper/1707.09074/) (2017)
- [CBIR using features derived by Deep Learning](https://mlsanity.com/paper/2002.07877/) (2020)
- [Semantic Image Retrieval by Uniting Deep Neural Networks and Cognitive Architectures](https://mlsanity.com/paper/1806.06946/) (2018)
- [Approximate Query Matching for Image Retrieval](https://mlsanity.com/paper/1803.05401/) (2018)
- [CNN-VWII: An Efficient Approach for Large-Scale Video Retrieval by Image Queries](https://mlsanity.com/paper/1810.06030/) (2018)
- [A Fast Content-Based Image Retrieval Method Using Deep Visual Features](https://mlsanity.com/paper/1908.01505/) (2019)

## Abstract (arXiv)

We propose a novel, efficient, modular and scalable framework for content based visual media retrieval systems by leveraging the power of Deep Learning which is flexible to work both for images and videos conjointly and we also introduce an efficient comparison and filtering metric for retrieval. We put forward our findings from critical performance tests comparing our method to the predominant conventional approach to demonstrate the feasibility and efficiency of the proposed solution with best practices, possible improvements that may further augment the ability of retrieval architectures.