# A Hybrid Spatial-temporal Deep Learning Architecture for Lane Detection

arXiv: https://arxiv.org/abs/2110.04079 · HTML: https://mlsanity.com/paper/2110.04079/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- Builds on 24 of its 61 references that are in the map.
- First posted on arXiv 5 Oct 2021.

## Builds on (most cited first, 24 of 24)

- [U-Net: Convolutional Networks for Biomedical Image Segmentation](https://mlsanity.com/paper/1505.04597/) (2015)
- [Sequence to Sequence Learning with Neural Networks](https://mlsanity.com/paper/1409.3215/) (2014)
- [SegNet: A Deep Convolutional Encoder-Decoder Architecture for Image Segmentation](https://mlsanity.com/paper/1511.00561/) (2015)
- [Convolutional LSTM Network: A Machine Learning Approach for Precipitation Nowcasting](https://mlsanity.com/paper/1506.04214/) (2015)
- [On the Variance of the Adaptive Learning Rate and Beyond](https://mlsanity.com/paper/1908.03265/) (2019)
- [Spatial As Deep: Spatial CNN for Traffic Scene Understanding](https://mlsanity.com/paper/1712.06080/) (2017)
- [Delving Deeper into Convolutional Networks for Learning Video Representations](https://mlsanity.com/paper/1511.06432/) (2015)
- [Keep your Eyes on the Lane: Real-time Attention-guided Lane Detection](https://mlsanity.com/paper/2010.12035/) (2020)
- [Towards End-to-End Lane Detection: an Instance Segmentation Approach](https://mlsanity.com/paper/1802.05591/) (2018)
- [Ultra Fast Structure-aware Deep Lane Detection](https://mlsanity.com/paper/2004.11757/) (2020)
- [End-to-end Lane Shape Prediction with Transformers](https://mlsanity.com/paper/2011.04233/) (2020)
- [PolyLaneNet: Lane Estimation via Deep Polynomial Regression](https://mlsanity.com/paper/2004.10924/) (2020)
- [CondLaneNet: a Top-to-down Lane Detection Framework Based on Conditional Convolution](https://mlsanity.com/paper/2105.05003/) (2021)
- [CurveLane-NAS: Unifying Lane-Sensitive Architecture Search and Adaptive Point Blending](https://mlsanity.com/paper/2007.12147/) (2020)
- [Key Points Estimation and Point Instance Segmentation Approach for Lane Detection](https://mlsanity.com/paper/2002.06604/) (2020)
- [Inter-Region Affinity Distillation for Road Marking Segmentation](https://mlsanity.com/paper/2004.05304/) (2020)
- [End-to-End Lane Marker Detection via Row-wise Classification](https://mlsanity.com/paper/2005.08630/) (2020)
- [Real time Detection of Lane Markers in Urban Streets](https://mlsanity.com/paper/1411.7113/) (2014)
- [Robust Lane Detection from Continuous Driving Scenes Using Deep Neural Networks](https://mlsanity.com/paper/1903.02193/) (2019)
- [FastDraw: Addressing the Long Tail of Lane Detection by Adapting a Sequential Prediction Network](https://mlsanity.com/paper/1905.04354/) (2019)
- [Fast Scene Understanding for Autonomous Driving](https://mlsanity.com/paper/1708.02550/) (2017)
- [Lane Detection in Low-light Conditions Using an Efficient Data Enhancement : Light Conditions Style Transfer](https://mlsanity.com/paper/2002.01177/) (2020)
- [Robust Gait Recognition by Integrating Inertial and RGBD Sensors](https://mlsanity.com/paper/1610.09816/) (2016)
- [Ego-Lane Analysis System (ELAS): Dataset and Algorithms](https://mlsanity.com/paper/1806.05984/) (2018)

## Built on by (most cited first, 5 of 5)

- [Lane Detection with Versatile AtrousFormer and Local Semantic Guidance](https://mlsanity.com/paper/2203.04067/) (2022)
- [Robust Lane Detection through Self Pre-training with Masked Sequential Autoencoders and Fine-tuning with Customized PolyLoss](https://mlsanity.com/paper/2305.17271/) (2023)
- [Efficient Sequential Neural Network with Spatial-Temporal Attention and Linear LSTM for Robust Lane Detection Using Multi-Frame Images](https://mlsanity.com/paper/2602.03669/) (2026)
- [CNN based Intelligent Streetlight Management Using Smart CCTV Camera and Semantic Segmentation](https://mlsanity.com/paper/2209.08633/) (2022)
- [Structure-Enhanced Features and Quality-Aware Dynamic Anchor Scoring for Robust Lane Detection](https://mlsanity.com/paper/2608.09610/) (2026)

## Abstract (arXiv)

Accurate and reliable lane detection is vital for the safe performance of lane-keeping assistance and lane departure warning systems. However, under certain challenging circumstances, it is difficult to get satisfactory performance in accurately detecting the lanes from one single image as mostly done in current literature. Since lane markings are continuous lines, the lanes that are difficult to be accurately detected in the current single image can potentially be better deduced if information from previous frames is incorporated. This study proposes a novel hybrid spatial-temporal (ST) sequence-to-one deep learning architecture. This architecture makes full use of the ST information in multiple continuous image frames to detect the lane markings in the very last frame. Specifically, the hybrid model integrates the following aspects: (a) the single image feature extraction module equipped with the spatial convolutional neural network; (b) the ST feature integration module constructed by ST recurrent neural network; (c) the encoder-decoder structure, which makes this image segmentation problem work in an end-to-end supervised learning format. Extensive experiments reveal that the proposed model architecture can effectively handle challenging driving scenes and outperforms available state-of-the-art methods.