Visual Attention in Imaginative Agents
Not on the map. Why?
Abstract (arXiv)
We present a recurrent agent who perceives surroundings through a series of discrete fixations. At each timestep, the agent imagines a variety of plausible scenes consistent with the fixation history. The next fixation is planned using uncertainty in the content of the imagined scenes. As time progresses, the agent becomes more certain about the content of the surrounding, and the variety in the imagined scenes reduces. The agent is built using a variational autoencoder and normalizing flows, and trained in an unsupervised manner on a proxy task of scene-reconstruction. The latent representations of the imagined scenes are found to be useful for performing pixel-level and scene-level tasks by higher-order modules. The agent is tested on various 2D and 3D datasets.
Key facts. Cited by 2 papers in the map (data through 7 Oct 2026). Builds on 42 of its 80 references that are in the map. First posted on arXiv 1 Apr 2021.
Builds on
42 of 80 references.
References · 42
- 2016The Cityscapes Dataset for Semantic Urban Scene Understandingcited by 4,427
- 2015Variational Inference with Normalizing Flowscited by 2,161
- 2016Density estimation using Real NVPcited by 2,029
- 2018Unsupervised Representation Learning by Predicting Image Rotationscited by 1,586
- 2018Glow: Generative Flow with Invertible 1x1 Convolutionscited by 1,539
- 2015SegNet: A Deep Convolutional Encoder-Decoder Architecture for Image Segmentationcited by 1,483
- 2015Show, Attend and Tell: Neural Image Caption Generation with Visual Attentioncited by 1,421
- 2014NICE: Non-linear Independent Components Estimationcited by 1,052
- 2016Unsupervised Monocular Depth Estimation with Left-Right Consistencycited by 819
- 2017Masked Autoregressive Flow for Density Estimationcited by 680
- 2016Improving Variational Inference with Inverse Autoregressive Flowcited by 636
- 2018Generative Image Inpainting with Contextual Attentioncited by 578
30 more
- 2015Importance Weighted Autoencoderscited by 454
- 2017Residual Attention Network for Image Classificationcited by 385
- 2014Recurrent Models of Visual Attentioncited by 344
- 2016Ladder Variational Autoencoderscited by 315
- 2018A Probabilistic U-Net for Segmentation of Ambiguous Imagescited by 281
- 2016Unsupervised Learning for Physical Interaction through Video Predictioncited by 256
- 2016Split-Brain Autoencoders: Unsupervised Learning by Cross-Channel Predictioncited by 217
- 2015DRAW: A Recurrent Neural Network For Image Generationcited by 190
- 2017Colorization as a Proxy Task for Visual Understandingcited by 175
- 2015Attention to Scale: Scale-aware Semantic Image Segmentationcited by 169
- 2018Uncertainty Estimates and Multi-Hypotheses Networks for Optical Flowcited by 135
- 2018Inference Suboptimality in Variational Autoencoderscited by 131
- 2017Representation Learning by Learning to Countcited by 120
- 2016Learning in an Uncertain World: Representing Ambiguity Through Multiple Hypothesescited by 96
- 2016Stochastic Multiple Choice Learning for Training Diverse Deep Ensemblescited by 77
- 2016Recursive Recurrent Nets with Attention Modeling for OCR in the Wildcited by 74
- 2016Unsupervised Learning of 3D Structure from Imagescited by 67
- 2017Multi-Context Attention for Human Pose Estimationcited by 66
- 2018Learning Image Representations by Completing Damaged Jigsaw Puzzlescited by 66
- 2019Block Neural Autoregressive Flowcited by 62
- 2018Leveraging Unlabeled Data for Crowd Counting by Learning to Rankcited by 57
- 2017A Dataset for Developing and Benchmarking Active Visioncited by 50
- 2015Active Object Localization with Deep Reinforcement Learningcited by 42
- 2017Learning to Look Around: Intelligently Exploring Unseen Environments for Unknown Taskscited by 37
- 20143D ShapeNets: A Deep Representation for Volumetric Shapescited by 27
- 2016Look-ahead before you leap: end-to-end active recognition by forecasting the effect of motioncited by 17
- 2019Emergence of Exploratory Look-Around Behaviors through Active Observation Completioncited by 14
- 2015Learning Wake-Sleep Recurrent Attention Modelscited by 9
- 2019Where to Look Next: Unsupervised Active Visual Exploration on 360deg Inputcited by 5
- 2017Anticipating many futures: Online human motion prediction and synthesis for human-robot collaborationcited by 3
Built on by
Citing papers per year: 2023: 2.
2 papers.