# Semantic-embedded Similarity Prototype for Scene Recognition

arXiv: https://arxiv.org/abs/2308.05896 · HTML: https://mlsanity.com/paper/2308.05896/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 30 of its 56 references that are in the map.
- First posted on arXiv 11 Aug 2023.

## Builds on (most cited first, 30 of 30)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [Very Deep Convolutional Networks for Large-Scale Image Recognition](https://mlsanity.com/paper/1409.1556/) (2014)
- [Swin Transformer: Hierarchical Vision Transformer using Shifted Windows](https://mlsanity.com/paper/2103.14030/) (2021)
- [ImageNet Large Scale Visual Recognition Challenge](https://mlsanity.com/paper/1409.0575/) (2014)
- [Rethinking the Inception Architecture for Computer Vision](https://mlsanity.com/paper/1512.00567/) (2015)
- [A ConvNet for the 2020s](https://mlsanity.com/paper/2201.03545/) (2022)
- [Searching for MobileNetV3](https://mlsanity.com/paper/1905.02244/) (2019)
- [ShuffleNet V2: Practical Guidelines for Efficient CNN Architecture Design](https://mlsanity.com/paper/1807.11164/) (2018)
- [When Does Label Smoothing Help?](https://mlsanity.com/paper/1906.02629/) (2019)
- [MobileViT: Light-weight, General-purpose, and Mobile-friendly Vision Transformer](https://mlsanity.com/paper/2110.02178/) (2021)
- [Vision Transformer Adapter for Dense Predictions](https://mlsanity.com/paper/2205.08534/) (2022)
- [Deep Metric Learning via Lifted Structured Feature Embedding](https://mlsanity.com/paper/1511.06452/) (2015)
- [Deep metric learning using Triplet network](https://mlsanity.com/paper/1412.6622/) (2014)
- [Circle Loss: A Unified Perspective of Pair Similarity Optimization](https://mlsanity.com/paper/2002.10857/) (2020)
- [Training Deep Neural Networks on Noisy Labels with Bootstrapping](https://mlsanity.com/paper/1412.6596/) (2014)
- [Delving Deep into Label Smoothing](https://mlsanity.com/paper/2011.12562/) (2020)
- [Semantic-Aware Scene Recognition](https://mlsanity.com/paper/1909.02410/) (2019)
- [Scene recognition with CNNs: objects, scales and dataset bias](https://mlsanity.com/paper/1801.06867/) (2018)
- [Knowledge Guided Disambiguation for Large-Scale Scene Classification with Multi-Resolution CNNs](https://mlsanity.com/paper/1610.01119/) (2016)
- [Object-to-Scene: Learning to Transfer Object Knowledge to Indoor Scene Recognition](https://mlsanity.com/paper/2108.00399/) (2021)
- [DEDUCE: Diverse scEne Detection methods in Unseen Challenging Environments](https://mlsanity.com/paper/1908.00191/) (2019)
- [BORM: Bayesian Object Relation Model for Indoor Scene Recognition](https://mlsanity.com/paper/2108.00397/) (2021)
- [Scene Classification in Indoor Environments for Robots using Context Based Word Embeddings](https://mlsanity.com/paper/1908.06422/) (2019)
- [Hybrid CNN and Dictionary-Based Models for Scene Recognition and Domain Adaptation](https://mlsanity.com/paper/1601.07977/) (2016)
- [Reconstruction Regularized Deep Metric Learning for Multi-label Image Classification](https://mlsanity.com/paper/2007.13547/) (2020)
- [Network Pruning via Resource Reallocation](https://mlsanity.com/paper/2103.01847/) (2021)
- [SRRM: Semantic Region Relation Model for Indoor Scene Recognition](https://mlsanity.com/paper/2305.08540/) (2023)
- [Guided Deep Metric Learning](https://mlsanity.com/paper/2206.02029/) (2022)

## Abstract (arXiv)

Due to the high inter-class similarity caused by the complex composition and the co-existing objects across scenes, numerous studies have explored object semantic knowledge within scenes to improve scene recognition. However, a resulting challenge emerges as object information extraction techniques require heavy computational costs, thereby burdening the network considerably. This limitation often renders object-assisted approaches incompatible with edge devices in practical deployment. In contrast, this paper proposes a semantic knowledge-based similarity prototype, which can help the scene recognition network achieve superior accuracy without increasing the computational cost in practice. It is simple and can be plug-and-played into existing pipelines. More specifically, a statistical strategy is introduced to depict semantic knowledge in scenes as class-level semantic representations. These representations are used to explore correlations between scene classes, ultimately constructing a similarity prototype. Furthermore, we propose to leverage the similarity prototype to support network training from the perspective of Gradient Label Softening and Batch-level Contrastive Loss, respectively. Comprehensive evaluations on multiple benchmarks show that our similarity prototype enhances the performance of existing networks, all while avoiding any additional computational burden in practical deployments. Code and the statistical similarity prototype will be available at https://github.com/ChuanxinSong/SimilarityPrototype