# Speak2Label: Using Domain Knowledge for Creating a Large Scale Driver Gaze Zone Estimation Dataset

arXiv: https://arxiv.org/abs/2004.05973 · HTML: https://mlsanity.com/paper/2004.05973/

## Key facts

- Cited by 18 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 13 Apr 2020.

## Built on by (most cited first, 18 of 18)

- [A Survey on Autonomous Driving Datasets: Statistics, Annotation Quality, and a Future Outlook](https://mlsanity.com/paper/2401.01454/) (2024)
- [Automatic Gaze Analysis: A Survey of Deep Learning based Approaches](https://mlsanity.com/paper/2108.05479/) (2021)
- [What Do You See in Vehicle? Comprehensive Vision Solution for In-Vehicle Gaze Estimation](https://mlsanity.com/paper/2403.15664/) (2024)
- [A Review of Driver Gaze Estimation and Application in Gaze Behavior Understanding](https://mlsanity.com/paper/2307.01470/) (2023)
- [MTGLS: Multi-Task Gaze Estimation with Limited Supervision](https://mlsanity.com/paper/2110.12100/) (2021)
- [Multimodal Fusion Using Deep Learning Applied to Driver's Referencing of Outside-Vehicle Objects](https://mlsanity.com/paper/2107.12167/) (2021)
- [Leveraging Driver Field-of-View for Multimodal Ego-Trajectory Prediction](https://mlsanity.com/paper/2312.08558/) (2023)
- [AV-Gaze: A Study on the Effectiveness of Audio Guided Visual Attention Estimation for Non-Profilic Faces](https://mlsanity.com/paper/2207.03048/) (2022)
- [A Survey on Datasets for Decision-making of Autonomous Vehicle](https://mlsanity.com/paper/2306.16784/) (2023)
- [Explainable Interface for Human-Autonomy Teaming: A Survey](https://mlsanity.com/paper/2405.02583/) (2024)
- [myEye2Wheeler: A Two-Wheeler Indian Driver Real-World Eye-Tracking Dataset](https://mlsanity.com/paper/2502.12723/) (2025)
- [Using Visual and Vehicular Sensors for Driver Behavior Analysis: A Survey](https://mlsanity.com/paper/2308.13406/) (2023)
- [EraW-Net: Enhance-Refine-Align W-Net for Scene-Associated Driver Attention Estimation](https://mlsanity.com/paper/2408.08570/) (2024)
- [SGAP-Gaze: Scene Grid Attention Based Point-of-Gaze Estimation Network for Driver Gaze](https://mlsanity.com/paper/2604.19888/) (2026)
- ['Labelling the Gaps': A Weakly Supervised Automatic Eye Gaze Estimation](https://mlsanity.com/paper/2208.01840/) (2022)
- [RAZE: Region Guided Self-Supervised Gaze Representation Learning](https://mlsanity.com/paper/2208.02485/) (2022)
- [Occlusion-aware Driver Monitoring System using the Driver Monitoring Dataset](https://mlsanity.com/paper/2504.20677/) (2025)
- [TransGaze-Object: Transformer Based Driver Gaze Object Prediction Framework in Real Driving](https://mlsanity.com/paper/2609.10139/) (2026)

## Abstract (arXiv)

Labelling of human behavior analysis data is a complex and time consuming task. In this paper, a fully automatic technique for labelling an image based gaze behavior dataset for driver gaze zone estimation is proposed. Domain knowledge is added to the data recording paradigm and later labels are generated in an automatic manner using Speech To Text conversion (STT). In order to remove the noise in the STT process due to different illumination and ethnicity of subjects in our data, the speech frequency and energy are analysed. The resultant Driver Gaze in the Wild (DGW) dataset contains 586 recordings, captured during different times of the day including evenings. The large scale dataset contains 338 subjects with an age range of 18-63 years. As the data is recorded in different lighting conditions, an illumination robust layer is proposed in the Convolutional Neural Network (CNN). The extensive experiments show the variance in the dataset resembling real-world conditions and the effectiveness of the proposed CNN pipeline. The proposed network is also fine-tuned for the eye gaze prediction task, which shows the discriminativeness of the representation learnt by our network on the proposed DGW dataset. Project Page: https://sites.google.com/view/drivergazeprediction/home