# HawkDrive: A Transformer-driven Visual Perception System for Autonomous Driving in Night Scene

arXiv: https://arxiv.org/abs/2404.04653 · HTML: https://mlsanity.com/paper/2404.04653/

## Key facts

- Cited by 6 papers in the map (data through 14 Oct 2026).
- Placed in VLMs for autonomous driving, one of 9 directions in End-to-end autonomous driving.
- Builds on 15 of its 31 references that are in the map.
- First posted on arXiv 6 Apr 2024.

## Placed in

- End-to-end autonomous driving / [VLMs for autonomous driving](https://mlsanity.com/driving/vlms-for-autonomous-driving/)

## Builds on (most cited first, 15 of 15)

- [SegFormer: Simple and Efficient Design for Semantic Segmentation with Transformers](https://mlsanity.com/paper/2105.15203/) (2021)
- [Vision Transformers for Dense Prediction](https://mlsanity.com/paper/2103.13413/) (2021)
- [Unsupervised Image-to-Image Translation Networks](https://mlsanity.com/paper/1703.00848/) (2017)
- [EnlightenGAN: Deep Light Enhancement without Paired Supervision](https://mlsanity.com/paper/1906.06972/) (2019)
- [GMFlow: Learning Optical Flow via Global Matching](https://mlsanity.com/paper/2111.13680/) (2021)
- [Unifying Flow, Stereo and Depth Estimation](https://mlsanity.com/paper/2211.05783/) (2022)
- [Robot Operating System 2: Design, Architecture, and Uses In The Wild](https://mlsanity.com/paper/2211.07752/) (2022)
- [Night-to-Day Image Translation for Retrieval-based Localization](https://mlsanity.com/paper/1809.09767/) (2018)
- [From two rolling shutters to one global shutter](https://mlsanity.com/paper/2006.01964/) (2020)
- [On the Importance of Stereo for Accurate Depth Estimation: An Efficient Semi-Supervised Deep Neural Network Approach](https://mlsanity.com/paper/1803.09719/) (2018)
- [Improving Nighttime Driving-Scene Segmentation via Dual Image-adaptive Learnable Filters](https://mlsanity.com/paper/2207.01331/) (2022)
- [Image-to-Image Translation for Autonomous Driving from Coarsely-Aligned Image Pairs](https://mlsanity.com/paper/2209.11673/) (2022)
- [Impact of ROS 2 Node Composition in Robotic Systems](https://mlsanity.com/paper/2305.09933/) (2023)
- [Image-to-Image Translation-based Data Augmentation for Robust EV Charging Inlet Detection](https://mlsanity.com/paper/2112.05290/) (2021)
- [DarkSLAM: GAN-assisted Visual SLAM for Reliable Operation in Low-light Conditions](https://mlsanity.com/paper/2206.02199/) (2022)

## Built on by (most cited first, 6 of 6)

- [VLM-Auto: VLM-based Autonomous Driving Assistant with Human-like Behavior and Understanding for Complex Road Scenes](https://mlsanity.com/paper/2405.05885/) (2024)
- [VDT-Auto: End-to-end Autonomous Driving with VLM-Guided Diffusion Transformers](https://mlsanity.com/paper/2502.20108/) (2025)
- [UltraFast-LiNET: Light-weight multi-scale shift convolutional network for real-time low-light image enhancement](https://mlsanity.com/paper/2512.02965/) (2025)
- [From Darkness to Detail: Frequency-Aware SSMs for Low-Light Vision](https://mlsanity.com/paper/2408.09650/) (2024)
- [METDrive: Multi-modal End-to-end Autonomous Driving with Temporal Guidance](https://mlsanity.com/paper/2409.12667/) (2024)
- [NeoARCADE: Robust Calibration for Distance Estimation to Support Assistive Drones for the Visually Impaired](https://mlsanity.com/paper/2504.01988/) (2025)

## Abstract (arXiv)

Many established vision perception systems for autonomous driving scenarios ignore the influence of light conditions, one of the key elements for driving safety. To address this problem, we present HawkDrive, a novel perception system with hardware and software solutions. Hardware that utilizes stereo vision perception, which has been demonstrated to be a more reliable way of estimating depth information than monocular vision, is partnered with the edge computing device Nvidia Jetson Xavier AGX. Our software for low light enhancement, depth estimation, and semantic segmentation tasks, is a transformer-based neural network. Our software stack, which enables fast inference and noise reduction, is packaged into system modules in Robot Operating System 2 (ROS2). Our experimental results have shown that the proposed end-to-end system is effective in improving the depth estimation and semantic segmentation performance. Our dataset and codes will be released at https://github.com/ZionGo6/HawkDrive.