mlsanityData to 14 Oct 2026

Can LLMs Test Terminal User Interfaces?

arXiv 2608.03743 ↗First posted 4 Aug 2026cs.SE cs.AI cs.LGcited by 0

Not on the map. Why?

Abstract (arXiv)

Terminal User Interfaces (TUIs) combine the stateful, screen-oriented behaviour of GUIs with terminal deployment and are now common in developer tools. Yet they lack a dedicated testing methodology. We survey 197 real-world TUI applications: only 12% of test code exercises the interface, and 45% of those tests never send input, checking a static frame instead. We turn these applications into a headless benchmark spanning ratatui/Rust, bubbletea/Go, textual/Python, and ink/TypeScript, packaging each as an instrumented Docker image. We record line and widget coverage where reliable, rendered terminal states, and crashes. Under equal wall-clock budgets, we compare four frontier LLMs with random exploration. No model dominates. Random is a strong time-budgeted baseline, but its crash advantage comes from higher throughput: per interaction, LLM guidance is more efficient and uniquely reaches input-gated faults. Automatically deriving launch inputs yields the largest practical gain, enabling applications that otherwise never start. Line coverage poorly predicts crash discovery, weakening it as a proxy for test effectiveness. Automated TUI testing is feasible but far from solved, and honest baselines matter more than model choice. We release the coverage tool tuicov at https://github.com/tui-testing/tuicov and the testing framework tuibot at https://github.com/tui-testing/tuibot.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 10 of its 23 references that are in the map. First posted on arXiv 4 Aug 2026.

Builds on

10 of 23 references.

References · 10

  1. 2023An Empirical Evaluation of Using Large Language Models for Automated Unit Test Generationcited by 194
  2. 2022Large Language Models are Zero-Shot Fuzzers: Fuzzing Deep-Learning Libraries via Large Language Modelscited by 103
  3. 2023Is ChatGPT the Ultimate Programming Assistant -- How far is it?cited by 76
  4. 2023Make LLM a Testing Expert: Bringing Human-like Interaction to Mobile GUI Testing via Functionality-aware Decisionscited by 41
  5. 2025Guidelines for Empirical Studies in Software Engineering involving Large Language Modelscited by 21
  6. 2015Automated Test Input Generation for Android: Are We There Yet?cited by 10
  7. 2021Deep Reinforcement Learning for Black-Box Testing of Android Appscited by 10
  8. 2021An Empirical Study of Automated Unit Test Generation for Pythoncited by 10
  9. 2023Testing the Limits: Unusual Text Inputs Generation for Mobile App Crash Detection with Large Language Modelcited by 10
  10. 2021FLACOCO: Fault Localization for Java based on Industry-grade Coveragecited by 3

Built on by

Not cited yet.