mlsanityData to 7 Oct 2026

On depth prediction for autonomous driving using self-supervised learning

arXiv 2403.06194 ↗First posted 10 Mar 2024cs.CVcited by 0

Not on the map. Why?

Abstract (arXiv)

Perception of the environment is a critical component for enabling autonomous driving. It provides the vehicle with the ability to comprehend its surroundings and make informed decisions. Depth prediction plays a pivotal role in this process, as it helps the understanding of the geometry and motion of the environment. This thesis focuses on the challenge of depth prediction using monocular self-supervised learning techniques. The problem is approached from a broader perspective first, exploring conditional generative adversarial networks (cGANs) as a potential technique to achieve better generalization was performed. In doing so, a fundamental contribution to the conditional GANs, the acontrario cGAN was proposed. The second contribution entails a single image-to-depth self-supervised method, proposing a solution for the rigid-scene assumption using a novel transformer-based method that outputs a pose for each dynamic object. The third significant aspect involves the introduction of a video-to-depth map forecasting approach. This method serves as an extension of self-supervised techniques to predict future depths. This involves the creation of a novel transformer model capable of predicting the future depth of a given scene. Moreover, the various limitations of the aforementioned methods were addressed and a video-to-video depth maps model was proposed. This model leverages the spatio-temporal consistency of the input and output sequence to predict a more accurate depth sequence output. These methods have significant applications in autonomous driving (AD) and advanced driver assistance systems (ADAS).

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 67 of its 189 references that are in the map. First posted on arXiv 10 Mar 2024.

Builds on

67 of 189 references.

References · 67

  1. 2017Attention Is All You Needcited by 59,932
  2. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  3. 2014Adam: A Method for Stochastic Optimizationcited by 36,960
  4. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
  5. 2020Language Models are Few-Shot Learnerscited by 32,199
  6. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 29,915
  7. 2019PyTorch: An Imperative Style, High-Performance Deep Learning Librarycited by 20,160
  8. 2023GPT-4 Technical Reportcited by 18,132
  9. 2022Training language models to follow instructions with human feedbackcited by 15,712
  10. 2014Very Deep Convolutional Networks for Large-Scale Image Recognitioncited by 15,267
  11. 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,491
  12. 2021Swin Transformer: Hierarchical Vision Transformer using Shifted Windowscited by 12,082
38 more
  1. 2019EfficientNet: Rethinking Model Scaling for Convolutional Neural Networkscited by 6,415
  2. 2021Zero-Shot Text-to-Image Generationcited by 4,276
  3. 2021On the Opportunities and Risks of Foundation Modelscited by 3,961
  4. 2016Improved Techniques for Training GANscited by 2,938
  5. 2018Large Scale GAN Training for High Fidelity Natural Image Synthesiscited by 2,609
  6. 2021Masked-attention Mask Transformer for Universal Image Segmentationcited by 2,338
  7. 2014Conditional Generative Adversarial Netscited by 2,152
  8. 2020Training Generative Adversarial Networks with Limited Datacited by 1,184
  9. 2021Grounded Language-Image Pre-trainingcited by 1,147
  10. 2021Scaling Vision Transformerscited by 993
  11. 2016Generative Adversarial Text to Image Synthesiscited by 735
  12. 2022Vision Transformer Adapter for Dense Predictionscited by 564
  13. 2021MViTv2: Improved Multiscale Vision Transformers for Classification and Detectioncited by 516
  14. 2021TrackFormer: Multi-Object Tracking with Transformerscited by 515
  15. 2017Towards Principled Methods for Training Generative Adversarial Networkscited by 460
  16. 2016A Learned Representation For Artistic Stylecited by 340
  17. 2015Deep multi-scale video prediction beyond mean square errorcited by 297
  18. 2016Unsupervised Learning for Physical Interaction through Video Predictioncited by 256
  19. 2018cGANs with Projection Discriminatorcited by 253
  20. 2019Unsupervised Scale-consistent Depth and Ego-motion Learning from Monocular Videocited by 244
  21. 2017Stochastic Variational Video Predictioncited by 234
  22. 2017Geometric GANcited by 232
  23. 2017GANs Trained by a Two Time-Scale Update Rule Converge to a Local Nash Equilibriumcited by 208
  24. 2016Deep Predictive Coding Networks for Video Prediction and Unsupervised Learningcited by 203
  25. 2016Single-Image Depth Perception in the Wildcited by 179
  26. 2015ParseNet: Looking Wider to See Bettercited by 176
  27. 2017Modulating early visual processing by languagecited by 167
  28. 2020You Only Need Adversarial Supervision for Semantic Image Synthesiscited by 129
  29. 2019Learning to Predict Layout-to-image Conditional Convolutions for Semantic Image Synthesiscited by 108
  30. 2018Searching for Efficient Multi-Scale Architectures for Dense Image Predictioncited by 96
  31. 2017SfM-Net: Learning of Structure and Motion from Videocited by 89
  32. 2019VideoFlow: A Conditional Flow-Based Model for Stochastic Video Generationcited by 74
  33. 2021Are we ready for a new paradigm shift? A Survey on Visual Deep MLPcited by 63
  34. 2021Predicting Video with VQVAEcited by 61
  35. 2017Triple Generative Adversarial Netscited by 59
  36. 2021Learning Monocular Depth in Dynamic Scenes via Instance-Aware Projection Consistencycited by 58
  37. 2020SESAME: Semantic Editing of Scenes by Adding, Manipulating or Erasing Objectscited by 43
  38. 2021TransCenter: Transformers with Dense Representations for Multiple-Object Trackingcited by 42

50 most cited of 67 All 67 (JSON)

Built on by

Not cited yet.