# Federated Self-Supervised Learning of Monocular Depth Estimators for Autonomous Vehicles

arXiv: https://arxiv.org/abs/2310.04837 · HTML: https://mlsanity.com/paper/2310.04837/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 31 of its 60 references that are in the map.
- First posted on arXiv 7 Oct 2023.

## Builds on (most cited first, 30 of 31)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [U-Net: Convolutional Networks for Biomedical Image Segmentation](https://mlsanity.com/paper/1505.04597/) (2015)
- [Communication-Efficient Learning of Deep Networks from Decentralized Data](https://mlsanity.com/paper/1602.05629/) (2016)
- [Depth Map Prediction from a Single Image using a Multi-Scale Deep Network](https://mlsanity.com/paper/1406.2283/) (2014)
- [Unsupervised Monocular Depth Estimation with Left-Right Consistency](https://mlsanity.com/paper/1609.03677/) (2016)
- [Unsupervised Learning of Depth and Ego-Motion from Video](https://mlsanity.com/paper/1704.07813/) (2017)
- [Self-supervised Visual Feature Learning with Deep Neural Networks: A Survey](https://mlsanity.com/paper/1902.06162/) (2019)
- [3D Packing for Self-Supervised Monocular Depth Estimation](https://mlsanity.com/paper/1905.02693/) (2019)
- [Deeper Depth Prediction with Fully Convolutional Residual Networks](https://mlsanity.com/paper/1606.00373/) (2016)
- [Sparsity Invariant CNNs](https://mlsanity.com/paper/1708.06500/) (2017)
- [Unsupervised Scale-consistent Depth and Ego-motion Learning from Monocular Video](https://mlsanity.com/paper/1908.10553/) (2019)
- [Competitive Collaboration: Joint Unsupervised Learning of Depth, Camera Motion, Optical Flow and Motion Segmentation](https://mlsanity.com/paper/1805.09806/) (2018)
- [Rethinking Architecture Design for Tackling Data Heterogeneity in Federated Learning](https://mlsanity.com/paper/2106.06047/) (2021)
- [Federated Unsupervised Representation Learning](https://mlsanity.com/paper/2010.08982/) (2020)
- [FedProc: Prototypical Contrastive Federated Learning on Non-IID data](https://mlsanity.com/paper/2109.12273/) (2021)
- [Divergence-aware Federated Self-Supervised Learning](https://mlsanity.com/paper/2204.04385/) (2022)
- [Collaborative Unsupervised Visual Representation Learning from Decentralized Data](https://mlsanity.com/paper/2108.06492/) (2021)
- [Multi-Frame Self-Supervised Depth with Transformers](https://mlsanity.com/paper/2204.07616/) (2022)
- [SC-DepthV3: Robust Self-supervised Monocular Depth Estimation for Dynamic Scenes](https://mlsanity.com/paper/2211.03660/) (2022)
- [Opportunities of Federated Learning in Connected, Cooperative and Automated Industrial Systems](https://mlsanity.com/paper/2101.03367/) (2021)
- [Federated Self-Supervised Learning of Multi-Sensor Representations for Embedded Intelligence](https://mlsanity.com/paper/2007.13018/) (2020)
- [Auto-Rectify Network for Unsupervised Indoor Depth Estimation](https://mlsanity.com/paper/2006.02708/) (2020)
- [Deep Digging into the Generalization of Self-Supervised Monocular Depth Estimation](https://mlsanity.com/paper/2205.11083/) (2022)
- [Federated Self-supervised Learning for Heterogeneous Clients](https://mlsanity.com/paper/2205.12493/) (2022)
- [Decentralized Unsupervised Learning of Visual Representations](https://mlsanity.com/paper/2111.10763/) (2021)
- [Making a Case for Federated Learning in the Internet of Vehicles and Intelligent Transportation Systems](https://mlsanity.com/paper/2102.10142/) (2021)
- [A Survey on Deep Learning Architectures for Image-based Depth Reconstruction](https://mlsanity.com/paper/1906.06113/) (2019)
- [How deep should be the depth of convolutional neural networks: a backyard dog case study](https://mlsanity.com/paper/1805.01516/) (2018)
- [FedIL: Federated Incremental Learning from Decentralized Unlabeled Data with Convergence Analysis](https://mlsanity.com/paper/2302.11823/) (2023)
- [FedMAE: Federated Self-Supervised Learning with One-Block Masked Auto-Encoder](https://mlsanity.com/paper/2303.11339/) (2023)

## Abstract (arXiv)

Image-based depth estimation has gained significant attention in recent research on computer vision for autonomous vehicles in intelligent transportation systems. This focus stems from its cost-effectiveness and wide range of potential applications. Unlike binocular depth estimation methods that require two fixed cameras, monocular depth estimation methods only rely on a single camera, making them highly versatile. While state-of-the-art approaches for this task leverage self-supervised learning of deep neural networks in conjunction with tasks like pose estimation and semantic segmentation, none of them have explored the combination of federated learning and self-supervision to train models using unlabeled and private data captured by autonomous vehicles. The utilization of federated learning offers notable benefits, including enhanced privacy protection, reduced network consumption, and improved resilience to connectivity issues. To address this gap, we propose FedSCDepth, a novel method that combines federated learning and deep self-supervision to enable the learning of monocular depth estimators with comparable effectiveness and superior efficiency compared to the current state-of-the-art methods. Our evaluation experiments conducted on Eigen's Split of the KITTI dataset demonstrate that our proposed method achieves near state-of-the-art performance, with a test loss below 0.13 and requiring, on average, only 1.5k training steps and up to 0.415 GB of weight data transfer per autonomous vehicle on each round.