# CoDEPS: Online Continual Learning for Depth Estimation and Panoptic Segmentation

arXiv: https://arxiv.org/abs/2303.10147 · HTML: https://mlsanity.com/paper/2303.10147/

## Key facts

- Cited by 18 papers in the map (data through 14 Oct 2026).
- Builds on 29 of its 39 references that are in the map.
- First posted on arXiv 17 Mar 2023.

## Builds on (most cited first, 29 of 29)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [The Cityscapes Dataset for Semantic Urban Scene Understanding](https://mlsanity.com/paper/1604.01685/) (2016)
- [Gradient Episodic Memory for Continual Learning](https://mlsanity.com/paper/1706.08840/) (2017)
- [Unsupervised Monocular Depth Estimation with Left-Right Consistency](https://mlsanity.com/paper/1609.03677/) (2016)
- [Unsupervised Learning of Depth and Ego-Motion from Video](https://mlsanity.com/paper/1704.07813/) (2017)
- [KITTI-360: A Novel Dataset and Benchmarks for Urban Scene Understanding in 2D and 3D](https://mlsanity.com/paper/2109.13410/) (2021)
- [Gradient based sample selection for online continual learning](https://mlsanity.com/paper/1903.08671/) (2019)
- [Continual Test-Time Domain Adaptation](https://mlsanity.com/paper/2203.13591/) (2022)
- [Panoptic-DeepLab](https://mlsanity.com/paper/1910.04751/) (2019)
- [DAFormer: Improving Network Architectures and Training Strategies for Domain-Adaptive Semantic Segmentation](https://mlsanity.com/paper/2111.14887/) (2021)
- [DACS: Domain Adaptation via Cross-domain Mixed Sampling](https://mlsanity.com/paper/2007.08702/) (2020)
- [ClassMix: Segmentation-Based Data Augmentation for Semi-Supervised Learning](https://mlsanity.com/paper/2007.07936/) (2020)
- [D3VO: Deep Depth, Deep Pose and Deep Uncertainty for Monocular Visual Odometry](https://mlsanity.com/paper/2003.01060/) (2020)
- [ViP-DeepLab: Learning Visual Perception with Depth-aware Video Panoptic Segmentation](https://mlsanity.com/paper/2012.05258/) (2020)
- [Unsupervised Monocular Depth Learning in Dynamic Scenes](https://mlsanity.com/paper/2010.16404/) (2020)
- [Unsupervised Monocular Depth and Ego-motion Learning with Structure and Semantics](https://mlsanity.com/paper/1906.05717/) (2019)
- [Bird's-Eye-View Panoptic Segmentation Using Monocular Frontal View Images](https://mlsanity.com/paper/2108.03227/) (2021)
- [Cross-View Regularization for Domain Adaptive Panoptic Segmentation](https://mlsanity.com/paper/2103.02584/) (2021)
- [Amodal Panoptic Segmentation](https://mlsanity.com/paper/2202.11542/) (2022)
- [Self-Supervised Deep Visual Odometry with Online Adaptation](https://mlsanity.com/paper/2005.06136/) (2020)
- [Geometric Unsupervised Domain Adaptation for Semantic Segmentation](https://mlsanity.com/paper/2103.16694/) (2021)
- [ConfMix: Unsupervised Domain Adaptation for Object Detection via Confidence-based Mixing](https://mlsanity.com/paper/2210.11539/) (2022)
- [DESC: Domain Adaptation for Depth Estimation via Semantic Consistency](https://mlsanity.com/paper/2009.01579/) (2020)
- [Perceiving the Invisible: Proposal-Free Amodal Panoptic Segmentation](https://mlsanity.com/paper/2205.14637/) (2022)
- [Dynamic Object Removal and Spatio-Temporal RGB-D Inpainting via Geometry-Aware Adversarial Learning](https://mlsanity.com/paper/2008.05058/) (2020)
- [Unsupervised Domain Adaptation for LiDAR Panoptic Segmentation](https://mlsanity.com/paper/2109.15286/) (2021)
- [Continual SLAM: Beyond Lifelong Simultaneous Localization and Mapping Through Continual Learning](https://mlsanity.com/paper/2203.01578/) (2022)
- [Monocular Depth Estimation with Self-supervised Instance Adaptation](https://mlsanity.com/paper/2004.05821/) (2020)
- [Continual BatchNorm Adaptation (CBNA) for Semantic Segmentation](https://mlsanity.com/paper/2203.01074/) (2022)

## Built on by (most cited first, 18 of 18)

- [Cam4DOcc: Benchmark for Camera-Only 4D Occupancy Forecasting in Autonomous Driving Applications](https://mlsanity.com/paper/2311.17663/) (2023)
- [A Survey on Continual Semantic Segmentation: Theory, Challenge, Method and Application](https://mlsanity.com/paper/2310.14277/) (2023)
- [Few-Shot Panoptic Segmentation With Foundation Models](https://mlsanity.com/paper/2309.10726/) (2023)
- [Syn-Mediverse: A Multimodal Synthetic Dataset for Intelligent Scene Understanding of Healthcare Facilities](https://mlsanity.com/paper/2308.03193/) (2023)
- [A Point-Based Approach to Efficient LiDAR Multi-Task Perception](https://mlsanity.com/paper/2404.12798/) (2024)
- [A Good Foundation is Worth Many Labels: Label-Efficient Panoptic Segmentation](https://mlsanity.com/paper/2405.19035/) (2024)
- [Open-Set LiDAR Panoptic Segmentation Guided by Uncertainty-Aware Learning](https://mlsanity.com/paper/2506.13265/) (2025)
- [UnCLe: Benchmarking Unsupervised Continual Learning for Depth Completion](https://mlsanity.com/paper/2410.18074/) (2024)
- [On-Device Self-Supervised Learning of Low-Latency Monocular Depth from Only Events](https://mlsanity.com/paper/2412.06359/) (2024)
- [Latent Gaussian Splatting for 4D Panoptic Occupancy Tracking](https://mlsanity.com/paper/2602.23172/) (2026)
- [StreamTTO: Efficient Online Test-Time Optimization for Video Depth Completion](https://mlsanity.com/paper/2603.01765/) (2026)
- [Designing for Difference: How Human Characteristics Shape Perceptions of Collaborative Robots](https://mlsanity.com/paper/2507.16480/) (2025)
- [Panoptic-Depth Forecasting](https://mlsanity.com/paper/2409.12008/) (2024)
- [Label-Efficient LiDAR Panoptic Segmentation](https://mlsanity.com/paper/2503.02372/) (2025)
- [Learning Appearance and Motion Cues for Panoptic Tracking](https://mlsanity.com/paper/2503.09191/) (2025)
- [Class-Incremental Motion Forecasting](https://mlsanity.com/paper/2603.09420/) (2026)
- [DecTrain: Deciding When to Train a Monocular Depth DNN Online](https://mlsanity.com/paper/2410.02980/) (2024)
- [Hyp2Former: Hierarchy-Aware Hyperbolic Embeddings for Open-Set Panoptic Segmentation](https://mlsanity.com/paper/2605.02580/) (2026)

## Abstract (arXiv)

Operating a robot in the open world requires a high level of robustness with respect to previously unseen environments. Optimally, the robot is able to adapt by itself to new conditions without human supervision, e.g., automatically adjusting its perception system to changing lighting conditions. In this work, we address the task of continual learning for deep learning-based monocular depth estimation and panoptic segmentation in new environments in an online manner. We introduce CoDEPS to perform continual learning involving multiple real-world domains while mitigating catastrophic forgetting by leveraging experience replay. In particular, we propose a novel domain-mixing strategy to generate pseudo-labels to adapt panoptic segmentation. Furthermore, we explicitly address the limited storage capacity of robotic systems by leveraging sampling strategies for constructing a fixed-size replay buffer based on rare semantic class sampling and image diversity. We perform extensive evaluations of CoDEPS on various real-world datasets demonstrating that it successfully adapts to unseen environments without sacrificing performance on previous domains while achieving state-of-the-art results. The code of our work is publicly available at http://codeps.cs.uni-freiburg.de.