mlsanityData to 14 Oct 2026

See Where You Read with Eye Gaze Tracking and Large Language Model

arXiv 2409.19454 ↗First posted 28 Sep 2024cs.HC cs.AI cs.CVcited by 0

Not on the map. Why?

Abstract (arXiv)

Losing track of reading progress during line switching can be frustrating. Eye gaze tracking technology offers a potential solution by highlighting read paragraphs, aiding users in avoiding wrong line switches. However, the gap between gaze tracking accuracy (2-3 cm) and text line spacing (3-5 mm) makes direct application impractical. Existing methods leverage the linear reading pattern but fail during jump reading. This paper presents a reading tracking and highlighting system that supports both linear and jump reading. Based on experimental insights from the gaze nature study of 16 users, two gaze error models are designed to enable both jump reading detection and relocation. The system further leverages the large language model's contextual perception capability in aiding reading tracking. A reading tracking domain-specific line-gaze alignment opportunity is also exploited to enable dynamic and frequent calibration of the gaze results. Controlled experiments demonstrate reliable linear reading tracking, as well as 84% accuracy in tracking jump reading. Furthermore, real field tests with 18 volunteers demonstrated the system's effectiveness in tracking and highlighting read paragraphs, improving reading efficiency, and enhancing user experience.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 21 of its 51 references that are in the map. First posted on arXiv 28 Sep 2024.

Builds on

21 of 51 references.

References · 21

  1. 2017Attention Is All You Needcited by 60,361
  2. 2015Deep Residual Learning for Image Recognitioncited by 52,096
  3. 2023VPGTrans: Transfer Visual Prompt Generator across LLMscited by 80
  4. 2024Can we Soft Prompt LLMs for Graph Learning Tasks?cited by 52
  5. 2020Towards End-to-end Video-based Eye-Trackingcited by 39
  6. 2024Latency-Aware Generative Semantic Communications with Pre-Trained Diffusion Modelscited by 25
  7. 2024DrHouse: An LLM-empowered Diagnostic Reasoning System through Harnessing Outcomes from Sensor Data and Expert Knowledgecited by 24
  8. 2024Tapping the Potential of Large Language Models as Recommender Systems: A Comprehensive Framework and Empirical Analysiscited by 17
  9. 2024One Prompt To Rule Them All: LLMs for Opinion Summary Evaluationcited by 16
  10. 2023DVGaze: Dual-View Gaze Estimationcited by 15
  11. 2023Towards Green AI in Fine-tuning Large Language Models via Adaptive Backpropagationcited by 12
  12. 2023EFE: End-to-end Frame-to-Gaze Estimationcited by 11
9 more
  1. 2023The Age of Generative AI and AI-Generated Everythingcited by 7
  2. 2024Communication and Control Co-Design in 6G: Sequential Decision-Making with LLMscited by 6
  3. 2021Hidden Markov Modeling over Graphscited by 3
  4. 2022Continuous Gaze Tracking With Implicit Saliency-Aware Calibration on Mobile Devicescited by 3
  5. 2019Tracking the Progression of Reading Through Eye-gaze Measurementscited by 2
  6. 2019A Novel Slip-Kalman Filter to Track the Progression of Reading Through Eye-Gaze Measurementscited by 2
  7. 2019An Approach to Track Reading Progression Using Eye-Gaze Fixation Pointscited by 1
  8. 2023Learning Online Policies for Person Tracking in Multi-View Environmentscited by 1
  9. 2024Input Conditioned Graph Generation for Language Agentscited by 1

Built on by

Not cited yet.