# MAMo: Leveraging Memory and Attention for Monocular Video Depth Estimation

arXiv: https://arxiv.org/abs/2307.14336 · HTML: https://mlsanity.com/paper/2307.14336/

## Key facts

- Cited by 24 papers in the map (data through 14 Oct 2026).
- Builds on 54 of its 77 references that are in the map.
- First posted on arXiv 26 Jul 2023.

## Builds on (most cited first, 30 of 54)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Swin Transformer: Hierarchical Vision Transformer using Shifted Windows](https://mlsanity.com/paper/2103.14030/) (2021)
- [EfficientNet: Rethinking Model Scaling for Convolutional Neural Networks](https://mlsanity.com/paper/1905.11946/) (2019)
- [SegFormer: Simple and Efficient Design for Semantic Segmentation with Transformers](https://mlsanity.com/paper/2105.15203/) (2021)
- [RAFT: Recurrent All-Pairs Field Transforms for Optical Flow](https://mlsanity.com/paper/2003.12039/) (2020)
- [Vision Transformers for Dense Prediction](https://mlsanity.com/paper/2103.13413/) (2021)
- [Linformer: Self-Attention with Linear Complexity](https://mlsanity.com/paper/2006.04768/) (2020)
- [Depth Map Prediction from a Single Image using a Multi-Scale Deep Network](https://mlsanity.com/paper/1406.2283/) (2014)
- [PWC-Net: CNNs for Optical Flow Using Pyramid, Warping, and Cost Volume](https://mlsanity.com/paper/1709.02371/) (2017)
- [A Large Dataset to Train Convolutional Networks for Disparity, Optical Flow, and Scene Flow Estimation](https://mlsanity.com/paper/1512.02134/) (2015)
- [Unsupervised Monocular Depth Estimation with Left-Right Consistency](https://mlsanity.com/paper/1609.03677/) (2016)
- [Deep Ordinal Regression Network for Monocular Depth Estimation](https://mlsanity.com/paper/1806.02446/) (2018)
- [Spatial Pyramid Pooling in Deep Convolutional Networks for Visual Recognition](https://mlsanity.com/paper/1406.4729/) (2014)
- [Neural Turing Machines](https://mlsanity.com/paper/1410.5401/) (2014)
- [AdaBins: Depth Estimation using Adaptive Bins](https://mlsanity.com/paper/2011.14141/) (2020)
- [3D Packing for Self-Supervised Monocular Depth Estimation](https://mlsanity.com/paper/1905.02693/) (2019)
- [A Unifying Review of Deep and Shallow Anomaly Detection](https://mlsanity.com/paper/2009.11732/) (2020)
- [Memory Networks](https://mlsanity.com/paper/1410.3916/) (2014)
- [DM-GAN: Dynamic Memory Generative Adversarial Networks for Text-to-Image Synthesis](https://mlsanity.com/paper/1904.01310/) (2019)
- [FlowFormer: A Transformer Architecture for Optical Flow](https://mlsanity.com/paper/2203.16194/) (2022)
- [The Temporal Opportunist: Self-Supervised Multi-Frame Monocular Depth](https://mlsanity.com/paper/2104.14540/) (2021)
- [BinsFormer: Revisiting Adaptive Bins for Monocular Depth Estimation](https://mlsanity.com/paper/2204.00987/) (2022)
- [Ask Me Anything: Dynamic Memory Networks for Natural Language Processing](https://mlsanity.com/paper/1506.07285/) (2015)
- [Revealing the Dark Secrets of Masked Image Modeling](https://mlsanity.com/paper/2205.13543/) (2022)
- [MeMOT: Multi-Object Tracking with Memory](https://mlsanity.com/paper/2203.16761/) (2022)
- [Attention Attention Everywhere: Monocular Depth Prediction with Skip Attention](https://mlsanity.com/paper/2210.09071/) (2022)
- [SimpleRecon: 3D Reconstruction Without 3D Convolutions](https://mlsanity.com/paper/2208.14743/) (2022)
- [Make-A-Story: Visual Memory Conditioned Consistent Story Generation](https://mlsanity.com/paper/2211.13319/) (2022)
- [Don't Forget The Past: Recurrent Depth Estimation from Monocular Video](https://mlsanity.com/paper/2001.02613/) (2020)
- [Neural RGB->D Sensing: Depth and Uncertainty from a Video Camera](https://mlsanity.com/paper/1901.02571/) (2019)

## Built on by (most cited first, 24 of 24)

- [Video Depth Anything: Consistent Depth Estimation for Super-Long Videos](https://mlsanity.com/paper/2501.12375/) (2025)
- [DepthCrafter: Generating Consistent Long Depth Sequences for Open-world Videos](https://mlsanity.com/paper/2409.02095/) (2024)
- [Align3R: Aligned Monocular Depth Estimation for Dynamic Videos](https://mlsanity.com/paper/2412.03079/) (2024)
- [GeometryCrafter: Consistent Geometry Estimation for Open-world Videos with Diffusion Priors](https://mlsanity.com/paper/2504.01016/) (2025)
- [Video Depth without Video Models](https://mlsanity.com/paper/2411.19189/) (2024)
- [Reconstructing 4D Spatial Intelligence: A Survey](https://mlsanity.com/paper/2507.21045/) (2025)
- [Harnessing Diffusion Models for Visual Perception with Meta Prompts](https://mlsanity.com/paper/2312.14733/) (2023)
- [FutureDepth: Learning to Predict the Future Improves Video Depth Estimation](https://mlsanity.com/paper/2403.12953/) (2024)
- [E3D-Bench: A Benchmark for End-to-End 3D Geometric Foundation Models](https://mlsanity.com/paper/2506.01933/) (2025)
- [StereoDiff: Stereo-Diffusion Synergy for Video Depth Estimation](https://mlsanity.com/paper/2506.20756/) (2025)
- [FiffDepth: Feed-forward Transformation of Diffusion-Based Generators for Detailed Depth Estimation](https://mlsanity.com/paper/2412.00671/) (2024)
- [Video Understanding: From Geometry and Semantics to Unified Models](https://mlsanity.com/paper/2603.17840/) (2026)
- [DeCoTR: Enhancing Depth Completion with 2D and 3D Attentions](https://mlsanity.com/paper/2403.12202/) (2024)
- [SVDC: Consistent Direct Time-of-Flight Video Depth Completion with Frequency Selective Fusion](https://mlsanity.com/paper/2503.01257/) (2025)
- [H3O: Hyper-Efficient 3D Occupancy Prediction with Heterogeneous Supervision](https://mlsanity.com/paper/2503.04059/) (2025)
- [Video Depth Propagation](https://mlsanity.com/paper/2512.10725/) (2025)
- [SciFlow: Empowering Lightweight Optical Flow Models with Self-Cleaning Iterations](https://mlsanity.com/paper/2404.08135/) (2024)
- [DepthSync: Diffusion Guidance-Based Depth Synchronization for Scale- and Geometry-Consistent Video Depth Estimation](https://mlsanity.com/paper/2507.01603/) (2025)
- [Towards Depth Foundation Model: Recent Trends in Vision-Based Depth Estimation](https://mlsanity.com/paper/2507.11540/) (2025)
- [STATIC : Surface Temporal Affine for TIme Consistency in Video Monocular Depth Estimation](https://mlsanity.com/paper/2412.01090/) (2024)
- [BePo: Dual Representation for 3D Occupancy Prediction](https://mlsanity.com/paper/2506.07002/) (2025)
- [Online Video Depth Anything: Temporally-Consistent Depth Prediction with Low Memory Consumption](https://mlsanity.com/paper/2510.09182/) (2025)
- [VDPP: Video Depth Post-Processing for Speed and Scalability](https://mlsanity.com/paper/2604.06665/) (2026)
- [GemDepth: Geometry-Embedded Features for 3D-Consistent Video Depth](https://mlsanity.com/paper/2605.10525/) (2026)

## Abstract (arXiv)

We propose MAMo, a novel memory and attention frame-work for monocular video depth estimation. MAMo can augment and improve any single-image depth estimation networks into video depth estimation models, enabling them to take advantage of the temporal information to predict more accurate depth. In MAMo, we augment model with memory which aids the depth prediction as the model streams through the video. Specifically, the memory stores learned visual and displacement tokens of the previous time instances. This allows the depth network to cross-reference relevant features from the past when predicting depth on the current frame. We introduce a novel scheme to continuously update the memory, optimizing it to keep tokens that correspond with both the past and the present visual information. We adopt attention-based approach to process memory features where we first learn the spatio-temporal relation among the resultant visual and displacement memory tokens using self-attention module. Further, the output features of self-attention are aggregated with the current visual features through cross-attention. The cross-attended features are finally given to a decoder to predict depth on the current frame. Through extensive experiments on several benchmarks, including KITTI, NYU-Depth V2, and DDAD, we show that MAMo consistently improves monocular depth estimation networks and sets new state-of-the-art (SOTA) accuracy. Notably, our MAMo video depth estimation provides higher accuracy with lower latency, when omparing to SOTA cost-volume-based video depth models.