# Can LLMs Test Terminal User Interfaces?

arXiv: https://arxiv.org/abs/2608.03743 · HTML: https://mlsanity.com/paper/2608.03743/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 10 of its 23 references that are in the map.
- First posted on arXiv 4 Aug 2026.

## Builds on (most cited first, 10 of 10)

- [An Empirical Evaluation of Using Large Language Models for Automated Unit Test Generation](https://mlsanity.com/paper/2302.06527/) (2023)
- [Large Language Models are Zero-Shot Fuzzers: Fuzzing Deep-Learning Libraries via Large Language Models](https://mlsanity.com/paper/2212.14834/) (2022)
- [Is ChatGPT the Ultimate Programming Assistant -- How far is it?](https://mlsanity.com/paper/2304.11938/) (2023)
- [Make LLM a Testing Expert: Bringing Human-like Interaction to Mobile GUI Testing via Functionality-aware Decisions](https://mlsanity.com/paper/2310.15780/) (2023)
- [Guidelines for Empirical Studies in Software Engineering involving Large Language Models](https://mlsanity.com/paper/2508.15503/) (2025)
- [Automated Test Input Generation for Android: Are We There Yet?](https://mlsanity.com/paper/1503.07217/) (2015)
- [Deep Reinforcement Learning for Black-Box Testing of Android Apps](https://mlsanity.com/paper/2101.02636/) (2021)
- [An Empirical Study of Automated Unit Test Generation for Python](https://mlsanity.com/paper/2111.05003/) (2021)
- [Testing the Limits: Unusual Text Inputs Generation for Mobile App Crash Detection with Large Language Model](https://mlsanity.com/paper/2310.15657/) (2023)
- [FLACOCO: Fault Localization for Java based on Industry-grade Coverage](https://mlsanity.com/paper/2111.12513/) (2021)

## Abstract (arXiv)

Terminal User Interfaces (TUIs) combine the stateful, screen-oriented behaviour of GUIs with terminal deployment and are now common in developer tools. Yet they lack a dedicated testing methodology. We survey 197 real-world TUI applications: only 12% of test code exercises the interface, and 45% of those tests never send input, checking a static frame instead. We turn these applications into a headless benchmark spanning ratatui/Rust, bubbletea/Go, textual/Python, and ink/TypeScript, packaging each as an instrumented Docker image. We record line and widget coverage where reliable, rendered terminal states, and crashes. Under equal wall-clock budgets, we compare four frontier LLMs with random exploration. No model dominates. Random is a strong time-budgeted baseline, but its crash advantage comes from higher throughput: per interaction, LLM guidance is more efficient and uniquely reaches input-gated faults. Automatically deriving launch inputs yields the largest practical gain, enabling applications that otherwise never start. Line coverage poorly predicts crash discovery, weakening it as a proxy for test effectiveness. Automated TUI testing is feasible but far from solved, and honest baselines matter more than model choice. We release the coverage tool tuicov at https://github.com/tui-testing/tuicov and the testing framework tuibot at https://github.com/tui-testing/tuibot.