mlsanityData to 14 Oct 2026

Bespoke Visual Assistance: What and How do Blind and Low-Vision People Create with Agentic Programming?

arXiv 2607.21760 ↗First posted 23 Jul 2026cs.HCcited by 0

On the map: LLM agents and tool use → GUI and mobile agents

Abstract (arXiv)

AI-powered assistive technologies have long supported blind and low vision (BLV) people in everyday tasks, but they are general-purpose and often fall short of meeting complex, individualized, in-situ accessibility needs. Though agentic programming tools, like GitHub Copilot, have the potential to bridge this gap by lowering the technical barriers to building personal AT using natural language, the practical applicability of this creation paradigm has been unknown. We address this knowledge gap through a two-phase longitudinal co-design study with five tech-savvy BLV users using ProgramAT, an agentic programming tool that supports the creation, iteration, and testing of camera-based AT. Overall, co-designers created over 37 tools, with some addressing needs unmet by any existing commercial AT such as identifying Uber rides or interpreting hand gestures. Qualitative feedback from our co-designers and analysis of development logs surface BLV strategies for tool creation, along with key challenges including model capability limits, specification conflicts, and barriers to successful creation. We discuss recommendations to provide appropriate conversational scaffolding, community tool sharing capabilities, and support for specialized models and personal datasets for future agentic programming environments to empower BLV users to create bespoke visual assistance for themselves.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in GUI and mobile agents, one of 16 directions in LLM agents and tool use. Builds on 32 of its 116 references that are in the map. First posted on arXiv 23 Jul 2026.

Builds on

32 of 116 references.

References · 32

  1. 2025SWE-PolyBench: A multi-language benchmark for repository level evaluation of coding agentscited by 85
  2. 2025Vibe Coding in Practice: Motivations, Challenges, and a Future Outlook -- a Grey Literature Reviewcited by 30
  3. 2024Misfitting With AI: How Blind People Verify and Contest AI Errorscited by 15
  4. 2026Programming by Chat: A Large-Scale Behavioral Analysis of 11,579 Real-World AI-Assisted IDE Sessionscited by 14
  5. 2024Improving Automated Code Reviews: Learning from Experiencecited by 13
  6. 2025Beyond Visual Perception: Insights from Smartphone Interaction of Visually Impaired Users with Large Multimodal Modelscited by 13
  7. 2025Vid2Coach: Transforming How-To Videos into Task Assistantscited by 12
  8. 2025Conversational AI as a Coding Assistant: Understanding Programmers' Interactions with and Expectations from Large Language Models for Codingcited by 11
  9. 2023Understanding Visual Arts Experiences of Blind Peoplecited by 9
  10. 2025CodeA11y: Making AI Coding Assistants Useful for Accessible Web Developmentcited by 8
  11. 2026DevBench: A Realistic, Developer-Informed Benchmark for Code Generation Modelscited by 8
  12. 2021Non-Visual Cooking: Exploring Practices and Challenges of Meal Preparation by People with Visual Impairmentscited by 7
20 more
  1. 2024ProgramAlly: Creating Custom Visual Access Programs via Multi-Modal End-User Programmingcited by 6
  2. 2025What do Blind and Low-Vision People Really Want from Assistive Smart Devices? Comparison of the Literature with a Focus Studycited by 6
  3. 2025"Before, I Asked My Mom, Now I Ask ChatGPT": Visual Privacy Management with Generative AI for Blind and Low-Vision Peoplecited by 6
  4. 2025Not Everyone Wins with LLMs: Behavioral Patterns and Pedagogical Implications for AI Literacy in Programmatic Data Sciencecited by 6
  5. 2026Understanding Dominant Themes in Reviewing Agentic AI-authored Codecited by 5
  6. 2023Mini-Giants: "Small" Language Models and Open Source Win-Wincited by 3
  7. 2025One Does Not Simply 'Mm-hmm': Exploring Backchanneling in the AAC Micro-Culturecited by 3
  8. 2024Why So Serious? Exploring Timely Humorous Comments in AAC Through AI-Powered Interfacescited by 2
  9. 2024ChartA11y: Designing Accessible Touch Experiences of Visualizations with Blind Smartphone Userscited by 2
  10. 2024Optimizing AI-Assisted Code Generationcited by 2
  11. 2025The Dilemma of Building Do-It-Yourself (DIY) Solutions for Workplace Accessibilitycited by 2
  12. 2025More than One Step at a Time: Designing Procedural Feedback for Non-visual Makeup Routinescited by 2
  13. 2025I, Robot? Exploring Ultra-Personalized AI-Powered AAC; an Autoethnographic Accountcited by 2
  14. 2025Smart Glasses for CVI: Co-Designing Extended Reality Solutions to Support Environmental Perception by People with Cerebral Visual Impairmentcited by 1
  15. 2025Helping or Homogenizing? GenAI as a Design Partner to Pre-Service SLPs for Just-in-Time Programming of AACcited by 1
  16. 2025SonoCraftAR: Towards Supporting Personalized Authoring of Sound-Reactive AR Interfaces by Deaf and Hard of Hearing Userscited by 1
  17. 2025How Far I'll Go: Imagining Futures of Conversational AI with People with Visual Impairments Through Design Fictioncited by 1
  18. 2025(R)evolution of Programming: Vibe Coding as a Post-Coding Paradigmcited by 1
  19. 2026Understanding Nature Engagement Experiences of Blind Peoplecited by 1
  20. 2026Three Modalities, Two Design Probes, One Prototype, and No Vision: Experience-Based Co-Design of a Multi-modal 3D Data Visualization Toolcited by 1

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2026Explainable AI for Blind and Low-Vision Users: Navigating Trust, Modality, and Interpretability in the Agentic Era
  2. 2026Position: Assistive Agents Need Accessibility Alignment
  3. 2024Emerging Practices for Large Multimodal Model (LMM) Assistance for People with Visual Impairments: Implications for Design
  4. 2026Making Videos Accessible for Blind and Low Vision Users Using a Multimodal Agent Video Player
  5. 2026LLM-Guided Agentic Floor Plan Parsing for Accessible Indoor Navigation of Blind and Low-Vision People
  6. 2026Are We There Yet? Assessing Computer-Use Agents for Blind Users' Accessible Interaction with Desktop Applications
  7. 2025Sighted by Default: Addressing Implicit Vision Assumptions in Real-Time VLM Assistance for BLV Users
  8. 2026"Zooming In" on Agentic Web Browsers as Assistive Technologies: A Case Study with a Low-Vision Technology Expert