The Single-File Test: A Longitudinal Public-Interface Evaluation of First-Output LLM Web Generation with Social Reach Tracking
Not on the map. Why?
Abstract (arXiv)
This paper presents an eight-week observational comparison of 68 single-file HTML generations collected across 17 public experiments in the "HTML AI Battle" project between December 10, 2025 and February 4, 2026. Four reasoning model families, GPT, Gemini, Grok, and Claude, were compared under a fixed public-interface protocol with no custom instructions, no personality tuning, and no repair prompts. Each output was evaluated from a rendered browser video using human scores and a Gemini LLM-as-a-judge layer for prompt adherence, functional correctness, and UI quality, then packaged into a standardized social-media protocol spanning X (Twitter), TikTok, and YouTube. The tracker was also used for two supervised predictive analyses: an experiment-level model for 24-hour X impressions and a generation-level model for HTML verbosity. Under this protocol, Claude was the strongest and most consistent family, leading mean performance and winning 9/17 prompts under the primary human weighted score. Longer measured reasoning time was not associated with higher quality overall. Gemini as a judge was significantly more lenient than the human evaluator on functional correctness and overall performance, while stable self-favoring bias remained unresolved. The exploratory X-impressions model remained weak under post-screen cross-validation (MAE = 46,874, R^2 = -0.377), whereas the HTML-lines model performed better, with a model-family-only baseline outperforming prompt-aware alternatives (MAE = 135.2, R^2 = 0.576). Overall, selected pre-publication technical/audio variables were not sufficient to predict 24-hour X reach, while code verbosity was driven much more by model family than by prompt wording. The comparisons remain observational and are limited by public-interface drift, access-path differences, and one primary human scorer.
Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 5 of its 11 references that are in the map. First posted on arXiv 6 May 2026.
Builds on
5 of 11 references.
References · 5
- 2023Judging LLM-as-a-Judge with MT-Bench and Chatbot Arenacited by 8,626
- 2021Evaluating Large Language Models Trained on Codecited by 8,320
- 2023Sparks of Artificial General Intelligence: Early experiments with GPT-4cited by 2,592
- 2023WebArena: A Realistic Web Environment for Building Autonomous Agentscited by 2,006
- 2023A Survey on Evaluation of Large Language Modelscited by 1,242
Built on by
Not cited yet.
Similar, not cited
Close in wording, no citation either way
- 2025Gender and content bias in Large Language Models: a case study on Google Gemini 2.0 Flash Experimental
- 2026Individual Turing Test: A Case Study of LLM-based Simulation Using Longitudinal Personal Data
- 2026The Judge Knows When It Knows: Calibrated Abstention for LLM-Based A/B-Test Prediction
- 2026CharacterFlywheel: Scaling Iterative Improvement of Engaging and Steerable LLMs in Production
- 2025Support Evaluation for the TREC 2024 RAG Track: Comparing Human versus LLM Judges
- 2026Thinking Less to Simulate Better: Intuitive Prompting Improves LLM Agents Simulating Individual Social Media Reactions, Including Unfamiliar Content
- 2025SVBench: Evaluation of Video Generation Models on Social Reasoning
- 2025CTTS: Collective Test-Time Scaling