# SaccadeCam: Adaptive Visual Attention for Monocular Depth Sensing

arXiv: https://arxiv.org/abs/2103.12981 · HTML: https://mlsanity.com/paper/2103.12981/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 25 of its 72 references that are in the map.
- First posted on arXiv 24 Mar 2021.

## Builds on (most cited first, 25 of 25)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Stand-Alone Self-Attention in Vision Models](https://mlsanity.com/paper/1906.05909/) (2019)
- [3D Packing for Self-Supervised Monocular Depth Estimation](https://mlsanity.com/paper/1905.02693/) (2019)
- [Sparsity Invariant CNNs](https://mlsanity.com/paper/1708.06500/) (2017)
- [DRAW: A Recurrent Neural Network For Image Generation](https://mlsanity.com/paper/1502.04623/) (2015)
- [Single-Image Depth Perception in the Wild](https://mlsanity.com/paper/1604.03901/) (2016)
- [Sparse-to-Dense: Depth Prediction from Sparse Depth Samples and a Single Image](https://mlsanity.com/paper/1709.07492/) (2017)
- [Self-supervised Monocular Trained Depth Estimation using Self-attention and Discrete Disparity Volume](https://mlsanity.com/paper/2003.13951/) (2020)
- [Zoom To Learn, Learn To Zoom](https://mlsanity.com/paper/1905.05169/) (2019)
- [Deep Depth Completion of a Single RGB-D Image](https://mlsanity.com/paper/1803.09326/) (2018)
- [Sparse and noisy LiDAR completion with RGB guidance and uncertainty](https://mlsanity.com/paper/1902.05356/) (2019)
- [Neural RGB->D Sensing: Depth and Uncertainty from a Video Camera](https://mlsanity.com/paper/1901.02571/) (2019)
- [Deep Optics for Monocular Depth Estimation and 3D Object Detection](https://mlsanity.com/paper/1904.08601/) (2019)
- [Deep Optics for Single-shot High-dynamic-range Imaging](https://mlsanity.com/paper/1908.00620/) (2019)
- [Learning Single Camera Depth Estimation using Dual-Pixels](https://mlsanity.com/paper/1904.05822/) (2019)
- [Learning When and Where to Zoom with Deep Reinforcement Learning](https://mlsanity.com/paper/2003.00425/) (2020)
- [Learning Sensor Multiplexing Design through Back-propagation](https://mlsanity.com/paper/1605.07078/) (2016)
- [Estimating Depth from RGB and Sparse Sensing](https://mlsanity.com/paper/1804.02771/) (2018)
- [Gated2Depth: Real-time Dense Lidar from Gated Images](https://mlsanity.com/paper/1902.04997/) (2019)
- [ATGV-Net: Accurate Depth Super-Resolution](https://mlsanity.com/paper/1607.07988/) (2016)
- [Efficient Object Detection in Large Images using Deep Reinforcement Learning](https://mlsanity.com/paper/1912.03966/) (2019)
- [Polka Lines: Learning Structured Illumination and Reconstruction for Active Stereo](https://mlsanity.com/paper/2011.13117/) (2020)
- [LiDAR-Flow: Dense Scene Flow Estimation from Sparse LiDAR and Stereo Images](https://mlsanity.com/paper/1910.14453/) (2019)
- [Hierarchical Attentive Recurrent Tracking](https://mlsanity.com/paper/1706.09262/) (2017)
- [RATM: Recurrent Attentive Tracking Model](https://mlsanity.com/paper/1510.08660/) (2015)

## Built on by (most cited first, 1 of 1)

- [SpectralZoom: Efficient Segmentation with an Adaptive Hyperspectral Camera](https://mlsanity.com/paper/2406.04287/) (2024)

## Abstract (arXiv)

Most monocular depth sensing methods use conventionally captured images that are created without considering scene content. In contrast, animal eyes have fast mechanical motions, called saccades, that control how the scene is imaged by the fovea, where resolution is highest. In this paper, we present the SaccadeCam framework for adaptively distributing resolution onto regions of interest in the scene. Our algorithm for adaptive resolution is a self-supervised network and we demonstrate results for end-to-end learning for monocular depth estimation. We also show preliminary results with a real SaccadeCam hardware prototype.