Lost in Instructions: Study of Blind Users' Experiences with DIY Manuals and AI-Rewritten Instructions for Assembly, Operation, and Troubleshooting of Tangible Products
Not on the map. Why?
Abstract (arXiv)
AI tools like ChatGPT and Be-My-AI are increasingly being used by blind individuals. Although prior work has explored their use in some Do-It-Yourself (DIY) tasks by blind individuals, little is known about how they use these tools and the available product-manual resources to assemble, operate, and troubleshoot physical or tangible products - tasks requiring spatial reasoning, structural understanding, and precise execution. We address this knowledge gap via an interview study and a usability study with blind participants, investigating how they leverage AI tools and product manuals for DIY tasks with physical products. Findings show that manuals are essential resources, but product-manual instructions are often inadequate for blind users. AI tools presently do not adequately address this insufficiency; in fact, we observed that they often exacerbate this issue with incomplete, incoherent, or misleading guidance. Lastly, we suggest improvements to AI tools for generating tailored instructions for blind users' DIY tasks involving tangible products.
Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 38 of its 134 references that are in the map. First posted on arXiv 20 Feb 2026.
Builds on
38 of 134 references.
References · 38
- 2020Language Models are Few-Shot Learnerscited by 32,199
- 2023GPT-4 Technical Reportcited by 18,132
- 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,302
- 2020Retrieval-Augmented Generation for Knowledge-Intensive NLP Taskscited by 9,160
- 2024SpatialRGPT: Grounded Spatial Reasoning in Vision Language Modelscited by 297
- 2024A Comprehensive Survey of Hallucination in Large Language, Image, Video and Audio Foundation Modelscited by 98
- 2024WorldScribe: Towards Context-Aware Live Visual Descriptionscited by 26
- 2024Investigating Use Cases of AI-Powered Scene Description Applications for Blind and Low Vision Peoplecited by 24
- 2023GenAssist: Making Image Generation Accessiblecited by 18
- 2025Morae: Proactively Pausing UI Agents for User Choicescited by 18
- 2024Misfitting With AI: How Blind People Verify and Contest AI Errorscited by 15
- 2024Making Short-Form Videos Accessible with Hierarchical Video Summariescited by 14
26 more
- 2025Beyond Visual Perception: Insights from Smartphone Interaction of Visually Impaired Users with Large Multimodal Modelscited by 13
- 2023VizAbility: Enhancing Chart Accessibility with LLM-based Conversational Interactioncited by 12
- 2025Vid2Coach: Transforming How-To Videos into Task Assistantscited by 12
- 2025Towards Understanding the Use of MLLM-Enabled Applications for Visual Interpretation by Blind and Low Vision Peoplecited by 11
- 2024"It's Kind of Context Dependent": Understanding Blind and Low Vision People's Video Accessibility Preferences Across Viewing Scenarioscited by 10
- 2024A Contextual Inquiry of People with Vision Impairments in Cookingcited by 9
- 2024Enabling Uniform Computer Interaction Experience for Blind Users through Large Language Modelscited by 9
- 2025VideoA11y: Method and Dataset for Accessible Video Descriptioncited by 8
- 2021Non-Visual Cooking: Exploring Practices and Challenges of Meal Preparation by People with Visual Impairmentscited by 7
- 2022"It Feels Like Taking a Gamble": Exploring Perceptions, Practices, and Challenges of Using Makeup and Cosmetics for People with Visual Impairmentscited by 7
- 2024A Recipe for Success? Exploring Strategies for Improving Non-Visual Access to Cooking Instructionscited by 7
- 2025CheckManual: A New Challenge and Benchmark for Manual-based Appliance Manipulationcited by 7
- 2021Improving the Accessibility of Scientific Documents: Current State, User Needs, and a System Solution to Enhance Scientific PDF Accessibility for Blind and Low Vision Userscited by 6
- 2024Understanding How Blind Users Handle Object Recognition Errors: Strategies and Challengescited by 5
- 2023SeeChart: Enabling Accessible Visualizations Through Interactive Natural Language Interface For People with Visual Impairmentscited by 4
- 2023TADA: Making Node-link Diagrams Accessible to Blind and Low-Vision Peoplecited by 4
- 2024MAIDR: Making Statistical Visualizations Accessible with Multimodal Data Representationcited by 4
- 2025DanmuA11y: Making Time-Synced On-Screen Video Comments (Danmu) Accessible to Blind and Low Vision Users via Multi-Viewer Audio Discussionscited by 4
- 2022Tactile Materials in Practice: Understanding the Experiences of Teachers of the Visually Impairedcited by 2
- 2022"I was Confused by It; It was Confused by Me:" Exploring the Experiences of People with Visual Impairments around Mobile Service Robotscited by 2
- 2023Assistive technology use in domestic activities by people who are blindcited by 2
- 2025AI-Powered Assistive Technologies for Visual Impairmentcited by 2
- 2025More than One Step at a Time: Designing Procedural Feedback for Non-visual Makeup Routinescited by 2
- 2025AROMA: Mixed-Initiative AI Assistance for Non-Visual Cooking by Grounding Multi-modal Information Between Reality and Videoscited by 2
- 2022Mathematical Content Browsing for Print-Disabled Readers Based on Virtual-World Exploration and Audio-Visual Sensory substitutioncited by 1
- 2024A Study of Digital Appliances Accessibility for People with Visual Disabilitiescited by 1
Built on by
Not cited yet.
Similar, not cited
Close in wording, no citation either way
- 2024Emerging Practices for Large Multimodal Model (LMM) Assistance for People with Visual Impairments: Implications for Design
- 2026Bespoke Visual Assistance: What and How do Blind and Low-Vision People Create with Agentic Programming?
- 2026Are We There Yet? Assessing Computer-Use Agents for Blind Users' Accessible Interaction with Desktop Applications
- 2026Explainable AI for Blind and Low-Vision Users: Navigating Trust, Modality, and Interpretability in the Agentic Era
- 2024ObjectFinder: An Open-Vocabulary Assistive System for Interactive Object Search by Blind People
- 2026A Multimodal Assistive System for Product Localization and Retrieval for People who are Blind or have Low Vision ↗
- 2026LLM-Guided Agentic Floor Plan Parsing for Accessible Indoor Navigation of Blind and Low-Vision People
- 2026Position: Assistive Agents Need Accessibility Alignment