# Leveraging Road Area Semantic Segmentation with Auxiliary Steering Task

arXiv: https://arxiv.org/abs/2212.09438 · HTML: https://mlsanity.com/paper/2212.09438/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 22 of its 28 references that are in the map.
- First posted on arXiv 19 Dec 2022.

## Builds on (most cited first, 22 of 22)

- [Feature Pyramid Networks for Object Detection](https://mlsanity.com/paper/1612.03144/) (2016)
- [The Cityscapes Dataset for Semantic Urban Scene Understanding](https://mlsanity.com/paper/1604.01685/) (2016)
- [Encoder-Decoder with Atrous Separable Convolution for Semantic Image Segmentation](https://mlsanity.com/paper/1802.02611/) (2018)
- [Deep High-Resolution Representation Learning for Visual Recognition](https://mlsanity.com/paper/1908.07919/) (2019)
- [End to End Learning for Self-Driving Cars](https://mlsanity.com/paper/1604.07316/) (2016)
- [Least Squares Generative Adversarial Networks](https://mlsanity.com/paper/1611.04076/) (2016)
- [Multi-Task Learning as Multi-Objective Optimization](https://mlsanity.com/paper/1810.04650/) (2018)
- [Learning to Adapt Structured Output Space for Semantic Segmentation](https://mlsanity.com/paper/1802.10349/) (2018)
- [Confidence Regularized Self-Training](https://mlsanity.com/paper/1908.09822/) (2019)
- [PAD-Net: Multi-Tasks Guided Prediction-and-Distillation Network for Simultaneous Depth Estimation and Scene Parsing](https://mlsanity.com/paper/1805.04409/) (2018)
- [Hierarchical Multi-Scale Attention for Semantic Segmentation](https://mlsanity.com/paper/2005.10821/) (2020)
- [MTI-Net: Multi-Scale Task Interaction Networks for Multi-Task Learning](https://mlsanity.com/paper/2001.06902/) (2020)
- [Domain Adaptation for Structured Output via Discriminative Patch Representations](https://mlsanity.com/paper/1901.05427/) (2019)
- [Semi-Supervised Semantic Segmentation with High- and Low-level Consistency](https://mlsanity.com/paper/1908.05724/) (2019)
- [Augmented Reality Meets Computer Vision : Efficient Data Generation for Urban Driving Scenes](https://mlsanity.com/paper/1708.01566/) (2017)
- [Canadian Adverse Driving Conditions Dataset](https://mlsanity.com/paper/2001.10117/) (2020)
- [Unsupervised Scene Adaptation with Memory Regularization in vivo](https://mlsanity.com/paper/1912.11164/) (2019)
- [Find Your Own Way: Weakly-Supervised Segmentation of Path Proposals for Urban Autonomy](https://mlsanity.com/paper/1610.01238/) (2016)
- [AuxNet: Auxiliary tasks enhanced Semantic Segmentation for Automated Driving](https://mlsanity.com/paper/1901.05808/) (2019)
- [A Fine-Grained Dataset and its Efficient Semantic Segmentation for Unstructured Driving Scenarios](https://mlsanity.com/paper/2103.13109/) (2021)
- [The NVIDIA PilotNet Experiments](https://mlsanity.com/paper/2010.08776/) (2020)
- [Multimodal End-to-End Learning for Autonomous Steering in Adverse Road and Weather Conditions](https://mlsanity.com/paper/2010.14924/) (2020)

## Abstract (arXiv)

Robustness of different pattern recognition methods is one of the key challenges in autonomous driving, especially when driving in the high variety of road environments and weather conditions, such as gravel roads and snowfall. Although one can collect data from these adverse conditions using cars equipped with sensors, it is quite tedious to annotate the data for training. In this work, we address this limitation and propose a CNN-based method that can leverage the steering wheel angle information to improve the road area semantic segmentation. As the steering wheel angle data can be easily acquired with the associated images, one could improve the accuracy of road area semantic segmentation by collecting data in new road environments without manual data annotation. We demonstrate the effectiveness of the proposed approach on two challenging data sets for autonomous driving and show that when the steering task is used in our segmentation model training, it leads to a 0.1-2.9% gain in the road area mIoU (mean Intersection over Union) compared to the corresponding reference transfer learning model.