What Else Needs Fixing? Exploring Cost-Effective Test-Time Compute for Revision Propagation in Artifacts Generated Through Conversation
On the map: LLM reasoning → Test-time compute and efficient thinking
Abstract (arXiv)
Large Language Models (LLMs) often help users generate artifacts through iterative cycles of generation and revision in conversation. A challenge here is that, when users specify only a local change during revision, LLMs must instead identify the relevant dependencies and propagate the revision to all affected parts of the artifact. This paper studies this ability of LLMs on conversationally generated artifacts, where the artifact context and its dependencies may be embedded in the conversation history. Toward practical use, we also explore cost-effective test-time compute for this new setting. Specifically, we introduce a new benchmark for this setting, and evaluate nine revision methods, including sequential reflection and parallel sampling variants, using gpt-oss-20b/120b, gpt-5.4-mini, and qwen3.5-9b/27b/122b on the benchmark. The results show that baselines achieve accuracies of 68.3--93%, and the most cost-effective method is selecting from three parallel samples using either LLM-based or medoid selection, which improves accuracy by 2.2--9.7%. Our code and dataset are available at https://github.com/ntt-dkiku/llm-revision-propagation.
Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in Test-time compute and efficient thinking, one of 17 directions in LLM reasoning. Builds on 3 of its 27 references that are in the map. First posted on arXiv 3 Sep 2026.
Builds on
3 of 27 references.
Closest ancestors · 1
Same direction, newest first
Other references · 2
- 2021Training Verifiers to Solve Math Word Problemscited by 10,109
- 2026EditPropBench: Measuring Factual Edit Propagation in Scientific Manuscriptscited by 1
Built on by
Not cited yet.
Similar, not cited
Close in wording, no citation either way
- 2026Lost in Conversation or Lost in Translation? Diagnosing Multi-Turn Degradation in RAG
- 2025Fidelity Before Structure: Verbatim Chunks Beat Lossy Artifact Extraction in Long-Conversation LLM Memory
- 2025ToolHaystack: Stress-Testing Tool-Augmented Language Models in Realistic Long-Term Interactions
- 2025Krul: Efficient State Restoration for Multi-turn Conversations with Dynamic Cross-layer KV Sharing
- 2025Benchmarking Failures in Tool-Augmented Language Models
- 2025MTRAG: A Multi-Turn Conversational Benchmark for Evaluating Retrieval-Augmented Generation Systems
- 2026RENDER: Controlling Reader-Facing Evidence in LLM Memory Evaluation
- 2023RELIC: Investigating Large Language Model Responses using Self-Consistency