# Visual Sensation and Perception Computational Models for Deep Learning: State of the art, Challenges and Prospects

arXiv: https://arxiv.org/abs/2109.03391 · HTML: https://mlsanity.com/paper/2109.03391/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 28 of its 134 references that are in the map.
- First posted on arXiv 8 Sep 2021.

## Builds on (most cited first, 28 of 28)

- [Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networks](https://mlsanity.com/paper/1703.03400/) (2017)
- [Bottom-Up and Top-Down Attention for Image Captioning and Visual Question Answering](https://mlsanity.com/paper/1707.07998/) (2017)
- [Recurrent Models of Visual Attention](https://mlsanity.com/paper/1406.6247/) (2014)
- [SCA-CNN: Spatial and Channel-wise Attention in Convolutional Networks for Image Captioning](https://mlsanity.com/paper/1611.05594/) (2016)
- [Attend, Infer, Repeat: Fast Scene Understanding with Generative Models](https://mlsanity.com/paper/1603.08575/) (2016)
- [Deep Multimodal Fusion by Channel Exchanging](https://mlsanity.com/paper/2011.05005/) (2020)
- [Visual Interaction Networks](https://mlsanity.com/paper/1706.01433/) (2017)
- [A Survey of Neuromorphic Computing and Neural Networks in Hardware](https://mlsanity.com/paper/1705.06963/) (2017)
- [Learning Video Object Segmentation with Visual Memory](https://mlsanity.com/paper/1704.05737/) (2017)
- [Learning Perceptual Inference by Contrasting](https://mlsanity.com/paper/1912.00086/) (2019)
- [Gated Feedback Recurrent Neural Networks](https://mlsanity.com/paper/1502.02367/) (2015)
- [Predicting the Computational Cost of Deep Learning Models](https://mlsanity.com/paper/1811.11880/) (2018)
- [Neuromorphic Deep Learning Machines](https://mlsanity.com/paper/1612.05596/) (2016)
- [Semi-Supervised Multi-Modal Multi-Instance Multi-Label Deep Network with Optimal Transport](https://mlsanity.com/paper/2104.08489/) (2021)
- [End-to-End Pixel-Based Deep Active Inference for Body Perception and Action](https://mlsanity.com/paper/2001.05847/) (2019)
- [DSTP-RNN: a dual-stage two-phase attention-based recurrent neural networks for long-term and multivariate time series prediction](https://mlsanity.com/paper/1904.07464/) (2019)
- [Deep Networks with Internal Selective Attention through Feedback Connections](https://mlsanity.com/paper/1407.3068/) (2014)
- [Top-Down Feedback for Crowd Counting Convolutional Neural Network](https://mlsanity.com/paper/1807.08881/) (2018)
- [Data Augmentation for Skin Lesion using Self-Attention based Progressive Generative Adversarial Network](https://mlsanity.com/paper/1910.11960/) (2019)
- [Brain Inspired Cognitive Model with Attention for Self-Driving Cars](https://mlsanity.com/paper/1702.05596/) (2017)
- [Defining Image Memorability using the Visual Memory Schema](https://mlsanity.com/paper/1903.02056/) (2019)
- [Attentional Neural Network: Feature Selection Using Cognitive Feedback](https://mlsanity.com/paper/1411.5140/) (2014)
- [SG-FCN: A Motion and Memory-Based Deep Learning Model for Video Saliency Detection](https://mlsanity.com/paper/1809.07988/) (2018)
- [X-CNN: Cross-modal Convolutional Neural Networks for Sparse Datasets](https://mlsanity.com/paper/1610.00163/) (2016)
- [Integrating both Visual and Audio Cues for Enhanced Video Caption](https://mlsanity.com/paper/1711.08097/) (2017)
- [A Neurally-Inspired Hierarchical Prediction Network for Spatiotemporal Sequence Learning and Prediction](https://mlsanity.com/paper/1901.09002/) (2019)
- [Salient Structure Detection by Context-Guided Visual Search](https://mlsanity.com/paper/1505.04364/) (2015)
- [Task-adaptive Asymmetric Deep Cross-modal Hashing](https://mlsanity.com/paper/2004.00197/) (2020)

## Abstract (arXiv)

Visual sensation and perception refers to the process of sensing, organizing, identifying, and interpreting visual information in environmental awareness and understanding. Computational models inspired by visual perception have the characteristics of complexity and diversity, as they come from many subjects such as cognition science, information science, and artificial intelligence. In this paper, visual perception computational models oriented deep learning are investigated from the biological visual mechanism and computational vision theory systematically. Then, some points of view about the prospects of the visual perception computational models are presented. Finally, this paper also summarizes the current challenges of visual perception and predicts its future development trends. Through this survey, it will provide a comprehensive reference for research in this direction.