mlsanityData to 7 Oct 2026

X2T: Training an X-to-Text Typing Interface with Online Learning from User Feedback

arXiv 2203.02072 ↗First posted 4 Mar 2022cs.HC cs.LGcited by 4

Not on the map. Why?

Abstract (arXiv)

We aim to help users communicate their intent to machines using flexible, adaptive interfaces that translate arbitrary user input into desired actions. In this work, we focus on assistive typing applications in which a user cannot operate a keyboard, but can instead supply other inputs, such as webcam images that capture eye gaze or neural activity measured by a brain implant. Standard methods train a model on a fixed dataset of user inputs, then deploy a static interface that does not learn from its mistakes; in part, because extracting an error signal from user behavior can be challenging. We investigate a simple idea that would enable such interfaces to improve over time, with minimal additional effort from the user: online learning from user feedback on the accuracy of the interface's actions. In the typing domain, we leverage backspaces as feedback that the interface did not perform the desired action. We propose an algorithm called x-to-text (X2T) that trains a predictive model of this feedback signal, and uses this model to fine-tune any existing, default interface for translating user input into actions that select words or characters. We evaluate X2T through a small-scale online user study with 12 participants who type sentences by gazing at their desired words, a large-scale observational study on handwriting samples from 60 users, and a pilot study with one participant using an electrocorticography-based brain-computer interface. The results show that X2T learns to outperform a non-adaptive default interface, stimulates user co-adaptation to the interface, personalizes the interface to individual users, and can leverage offline data collected from the default interface to improve its initial performance and accelerate online learning.

Key facts. Cited by 4 papers in the map (data through 7 Oct 2026). Builds on 25 of its 58 references that are in the map. First posted on arXiv 4 Mar 2022.

Builds on

25 of 58 references.

References · 25

  1. 2014Adam: A Method for Stochastic Optimizationcited by 36,960
  2. 2017Deep reinforcement learning from human preferencescited by 3,846
  3. 2019Transformer-XL: Attentive Language Models Beyond a Fixed-Length Contextcited by 1,830
  4. 2018Reinforcement Learning and Control as Probabilistic Inference: Tutorial and Reviewcited by 466
  5. 2013One Billion Word Benchmark for Measuring Progress in Statistical Language Modelingcited by 302
  6. 2016Eye Tracking for Everyonecited by 155
  7. 2017Interactive Learning from Policy-Dependent Human Feedbackcited by 155
  8. 2017Deep TAMER: Interactive Agent Shaping in High-Dimensional State Spacescited by 122
  9. 2016Sequence Tutor: Conservative Fine-Tuning of Sequence Generation Models with KL-controlcited by 100
  10. 2018Shared Autonomy via Deep Reinforcement Learningcited by 65
  11. 2019Deep Reinforcement Learning from Policy-Dependent Human Feedbackcited by 56
  12. 2016Learning Language Games through Interactioncited by 43
13 more
  1. 2020The EMPATHIC Framework for Task Learning from Implicit Human Feedbackcited by 38
  2. 2019Characterizing the dynamics of learning in repeated reference gamescited by 34
  3. 2020Shared Autonomy with Learned Latent Actionscited by 34
  4. 2020AvE: Assistance via Empowermentcited by 27
  5. 2020Residual Policy Learning for Shared Autonomycited by 25
  6. 2020Learning Adaptive Language Interfaces through Decompositioncited by 22
  7. 2017Human-Robot Mutual Adaptation in Shared Autonomycited by 16
  8. 2018Deep Contextual Multi-armed Banditscited by 16
  9. 2019Adaptive Robot-Assisted Feeding: An Online Learning Framework for Acquiring Previously Unseen Food Itemscited by 14
  10. 2018Learning Models for Shared Control of Human-Machine Systems with Unknown Dynamicscited by 11
  11. 2019Neural Contextual Bandits with UCB-based Explorationcited by 6
  12. 2015Neuroprosthetic decoder training as imitation learningcited by 4
  13. 2020Accelerating Reinforcement Learning Agent with EEG-based Implicit Human Feedbackcited by 4

Built on by

Cited by year

Citing papers per year: 2022: 3; 2023: 1.

4 papers.

  1. 2022Personalized Reward Learning with Interaction-Grounded Learning (IGL)cited by 9
  2. 2022First Contact: Unsupervised Human-Machine Co-Adaptation via Mutual Information Maximizationcited by 8
  3. 2022Interaction-Grounded Learning with Action-inclusive Feedbackcited by 7
  4. 2023Bootstrapping Adaptive Human-Machine Interfaces with Offline Reinforcement Learningcited by 0