mlsanityData to 14 Oct 2026

See Yourself in Others: Attending Multiple Tasks for Own Failure Detection

arXiv 2110.02549 ↗First posted 6 Oct 2021cs.CV cs.ROcited by 8

Not on the map. Why?

Abstract (arXiv)

Autonomous robots deal with unexpected scenarios in real environments. Given input images, various visual perception tasks can be performed, e.g., semantic segmentation, depth estimation and normal estimation. These different tasks provide rich information for the whole robotic perception system. All tasks have their own characteristics while sharing some latent correlations. However, some of the task predictions may suffer from the unreliability dealing with complex scenes and anomalies. We propose an attention-based failure detection approach by exploiting the correlations among multiple tasks. The proposed framework infers task failures by evaluating the individual prediction, across multiple visual perception tasks for different regions in an image. The formulation of the evaluations is based on an attention network supervised by multi-task uncertainty estimation and their corresponding prediction errors. Our proposed framework generates more accurate estimations of the prediction error for the different task's predictions.

Key facts. Cited by 8 papers in the map (data through 14 Oct 2026). Builds on 36 of its 48 references that are in the map. First posted on arXiv 6 Oct 2021.

Builds on

36 of 48 references.

References · 36

  1. 2017Attention Is All You Needcited by 60,361
  2. 2015Deep Residual Learning for Image Recognitioncited by 52,096
  3. 2017Mask R-CNNcited by 7,277
  4. 2018MobileNetV2: Inverted Residuals and Linear Bottleneckscited by 4,795
  5. 2015Dropout as a Bayesian Approximation: Representing Model Uncertainty in Deep Learningcited by 4,448
  6. 2016The Cityscapes Dataset for Semantic Urban Scene Understandingcited by 4,444
  7. 2016Simple and Scalable Predictive Uncertainty Estimation using Deep Ensemblescited by 3,926
  8. 2016A Baseline for Detecting Misclassified and Out-of-Distribution Examples in Neural Networkscited by 2,385
  9. 2017What Uncertainties Do We Need in Bayesian Deep Learning for Computer Vision?cited by 2,352
  10. 2017Multi-Task Learning Using Uncertainty to Weigh Losses for Scene Geometry and Semanticscited by 1,358
  11. 2018Deep Anomaly Detection with Outlier Exposurecited by 942
  12. 2016Unsupervised Monocular Depth Estimation with Left-Right Consistencycited by 820
24 more
  1. 2018Taskonomy: Disentangling Task Transfer Learningcited by 634
  2. 2016Cross-stitch Networks for Multi-task Learningcited by 467
  3. 2016Past, Present, and Future of Simultaneous Localization And Mapping: Towards the Robust-Perception Agecited by 411
  4. 2017Semantic Foggy Scene Understanding with Synthetic Datacited by 403
  5. 2019A Survey of Deep Learning Techniques for Autonomous Drivingcited by 403
  6. 2019Which Tasks Should Be Learned Together in Multi-task Learning?cited by 375
  7. 2019Enforcing geometric constraints of virtual normal for depth predictioncited by 234
  8. 2016UberNet: Training a `Universal' Convolutional Neural Network for Low-, Mid-, and High-Level Vision using Diverse Datasets and Limited Memorycited by 200
  9. 2020On the uncertainty of self-supervised monocular depth estimationcited by 182
  10. 2019Detecting the Unexpected via Image Resynthesiscited by 137
  11. 2018Uncertainty Estimates and Multi-Hypotheses Networks for Optical Flowcited by 135
  12. 2018Improving Semantic Segmentation via Video Propagation and Label Relaxationcited by 134
  13. 2020Semantically-Guided Representation Learning for Self-Supervised Monocular Depthcited by 134
  14. 2020EfficientPS: Efficient Panoptic Segmentationcited by 111
  15. 2021Pixel-wise Anomaly Detection in Complex Driving Scenescited by 109
  16. 2020Robust Learning Through Cross-Task Consistencycited by 105
  17. 2020Map-Guided Curriculum Domain Adaptation and Uncertainty-Aware Evaluation for Semantic Nighttime Image Segmentationcited by 86
  18. 2021Domain Adaptive Semantic Segmentation with Self-Supervised Depth Estimationcited by 80
  19. 2021Semantics for Robotic Mapping, Perception and Interaction: A Surveycited by 56
  20. 2018Prediction Error Meta Classification in Semantic Segmentation: Detection via Aggregated Dispersion Measures of Softmax Probabilitiescited by 54
  21. 2017Transitive Invariance for Self-supervised Visual Representation Learningcited by 37
  22. 2016Multi-Task Learning with Labeled and Unlabeled Taskscited by 17
  23. 2019This is not what I imagined: Error Detection for Semantic Segmentation through Visual Dissimilaritycited by 16
  24. 2021FSNet: A Failure Detection Framework for Semantic Segmentationcited by 9

Built on by

Cited by year

Citing papers per year: 2021: 1; 2022: 0; 2023: 5; 2024: 2.

8 papers.

  1. 2021Learning Multiple Dense Prediction Tasks from Partially Annotated Datacited by 42
  2. 2024Bootstrapping Reinforcement Learning with Imitation for Vision-Based Agile Flightcited by 31
  3. 2023Resilient Legged Local Navigation: Learning to Traverse with Compromised Perception End-to-Endcited by 23
  4. 2023Contrastive Learning for Enhancing Robust Scene Transfer in Vision-based Agile Flightcited by 15
  5. 2023Self-Aware Trajectory Prediction for Safe Autonomous Drivingcited by 5
  6. 2023Failure Detection for Motion Prediction of Autonomous Driving: An Uncertainty Perspectivecited by 4
  7. 2023High-Quality Unknown Object Instance Segmentation via Quadruple Boundary Error Refinementcited by 1
  8. 2024Towards Safe and Reliable Autonomous Driving: Dynamic Occupancy Set Predictioncited by 0