# GazeReader: Detecting Unknown Word Using Webcam for English as a Second Language (ESL) Learners

arXiv: https://arxiv.org/abs/2303.10443 · HTML: https://mlsanity.com/paper/2303.10443/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 4 of its 29 references that are in the map.
- First posted on arXiv 18 Mar 2023.

## Builds on (most cited first, 4 of 4)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [RoBERTa: A Robustly Optimized BERT Pretraining Approach](https://mlsanity.com/paper/1907.11692/) (2019)
- [Entity Recognition at First Sight: Improving NER with Eye Movement Information](https://mlsanity.com/paper/1902.10068/) (2019)
- [FaceOri: Tracking Head Position and Orientation Using Ultrasonic Ranging on Earphones](https://mlsanity.com/paper/2203.10553/) (2022)

## Built on by (most cited first, 2 of 2)

- [G-VOILA: Gaze-Facilitated Information Querying in Daily Scenarios](https://mlsanity.com/paper/2405.07652/) (2024)
- [Gaze-Driven Sentence Simplification for Language Learners: Enhancing Comprehension and Readability](https://mlsanity.com/paper/2310.00355/) (2023)

## Abstract (arXiv)

Automatic unknown word detection techniques can enable new applications for assisting English as a Second Language (ESL) learners, thus improving their reading experiences. However, most modern unknown word detection methods require dedicated eye-tracking devices with high precision that are not easily accessible to end-users. In this work, we propose GazeReader, an unknown word detection method only using a webcam. GazeReader tracks the learner's gaze and then applies a transformer-based machine learning model that encodes the text information to locate the unknown word. We applied knowledge enhancement including term frequency, part of speech, and named entity recognition to improve the performance. The user study indicates that the accuracy and F1-score of our method were 98.09% and 75.73%, respectively. Lastly, we explored the design scope for ESL reading and discussed the findings.