# ARTiST: Automated Text Simplification for Task Guidance in Augmented Reality

arXiv: https://arxiv.org/abs/2402.18797 · HTML: https://mlsanity.com/paper/2402.18797/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 53 of its 317 references that are in the map.
- First posted on arXiv 29 Feb 2024.

## Builds on (most cited first, 30 of 53)

- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer](https://mlsanity.com/paper/1910.10683/) (2019)
- [RoBERTa: A Robustly Optimized BERT Pretraining Approach](https://mlsanity.com/paper/1907.11692/) (2019)
- [On Calibration of Modern Neural Networks](https://mlsanity.com/paper/1706.04599/) (2017)
- [A Survey on Bias and Fairness in Machine Learning](https://mlsanity.com/paper/1908.09635/) (2019)
- [Calibrate Before Use: Improving Few-Shot Performance of Language Models](https://mlsanity.com/paper/2102.09690/) (2021)
- [Energy and Policy Considerations for Deep Learning in NLP](https://mlsanity.com/paper/1906.02243/) (2019)
- [Language (Technology) is Power: A Critical Survey of "Bias" in NLP](https://mlsanity.com/paper/2005.14050/) (2020)
- [The State and Fate of Linguistic Diversity and Inclusion in the NLP World](https://mlsanity.com/paper/2004.09095/) (2020)
- [Detecting Twenty-thousand Classes using Image-level Supervision](https://mlsanity.com/paper/2201.02605/) (2022)
- [CoAuthor: Designing a Human-AI Collaborative Writing Dataset for Exploring Language Model Capabilities](https://mlsanity.com/paper/2201.06796/) (2022)
- [Examining Gender and Race Bias in Two Hundred Sentiment Analysis Systems](https://mlsanity.com/paper/1805.04508/) (2018)
- [Challenges and Strategies in Cross-Cultural NLP](https://mlsanity.com/paper/2203.10020/) (2022)
- [The Unreliability of Explanations in Few-shot Prompting for Textual Reasoning](https://mlsanity.com/paper/2205.03401/) (2022)
- [Towards a Critical Race Methodology in Algorithmic Fairness](https://mlsanity.com/paper/1912.03593/) (2019)
- [Understanding and Evaluating Racial Biases in Image Captioning](https://mlsanity.com/paper/2106.08503/) (2021)
- [Neural CRF Model for Sentence Alignment in Text Simplification](https://mlsanity.com/paper/2005.02324/) (2020)
- [Language Models that Seek for Knowledge: Modular Search & Generation for Dialogue and Prompt Completion](https://mlsanity.com/paper/2203.13224/) (2022)
- [Discrimination in Online Ad Delivery](https://mlsanity.com/paper/1301.6822/) (2013)
- [Human-centered Explainable AI: Towards a Reflective Sociotechnical Approach](https://mlsanity.com/paper/2002.01092/) (2020)
- [The Fallacy of AI Functionality](https://mlsanity.com/paper/2206.09511/) (2022)
- [Do Datasets Have Politics? Disciplinary Values in Computer Vision Dataset Development](https://mlsanity.com/paper/2108.04308/) (2021)
- [Quantifying Social Biases in NLP: A Generalization and Empirical Comparison of Extrinsic Fairness Metrics](https://mlsanity.com/paper/2106.14574/) (2021)
- [Iteratively Prompt Pre-trained Language Models for Chain of Thought](https://mlsanity.com/paper/2203.08383/) (2022)
- [Perturbation Sensitivity Analysis to Detect Unintended Model Biases](https://mlsanity.com/paper/1910.04210/) (2019)
- [Racial Disparity in Natural Language Processing: A Case Study of Social Media African-American English](https://mlsanity.com/paper/1707.00061/) (2017)
- [Re-contextualizing Fairness in NLP: The Case of India](https://mlsanity.com/paper/2209.12226/) (2022)
- [Nationality Bias in Text Generation](https://mlsanity.com/paper/2302.02463/) (2023)
- [Knowing More About Questions Can Help: Improving Calibration in Question Answering](https://mlsanity.com/paper/2106.01494/) (2021)
- [Getting MoRE out of Mixture of Language Model Reasoning Experts](https://mlsanity.com/paper/2305.14628/) (2023)

## Abstract (arXiv)

Text presented in augmented reality provides in-situ, real-time information for users. However, this content can be challenging to apprehend quickly when engaging in cognitively demanding AR tasks, especially when it is presented on a head-mounted display. We propose ARTiST, an automatic text simplification system that uses a few-shot prompt and GPT-3 models to specifically optimize the text length and semantic content for augmented reality. Developed out of a formative study that included seven users and three experts, our system combines a customized error calibration model with a few-shot prompt to integrate the syntactic, lexical, elaborative, and content simplification techniques, and generate simplified AR text for head-worn displays. Results from a 16-user empirical study showed that ARTiST lightens the cognitive load and improves performance significantly over both unmodified text and text modified via traditional methods. Our work constitutes a step towards automating the optimization of batch text data for readability and performance in augmented reality.