mlsanityData to 7 Oct 2026

FrankenReport: Early Exiting in Long-Form Generation Using Expected Value of Computation

arXiv 2609.06320 ↗First posted 6 Sep 2026cs.HC cs.CLcited by 0

On the map: Efficient LLM inference and serving → Efficient reasoning and agent inference

Abstract (arXiv)

While deep research systems address interactive information-seeking needs impressively, their real-world deployments face latency and resource-consumption challenges. We present FrankenReport, an interface for long-form knowledge-seeking report generation that supports adaptive early exiting per section: it evaluates intermediate outputs during generation and predicts whether further targeted computation will yield significant quality gains. In a simulation study, FrankenReport outperforms random allocation baselines by a large margin (up to 4x) under low budgets and smoothly recovers full-pipeline quality as the budget grows, showing that future quality gains are predictable from intermediate drafts. Through experiments and user studies, we further show that despite varying preferences across users and topics, FrankenReport adapts to simple, natural user feedback as efficiently as methods requiring much costlier supervision such as generated drafts and explicit rationales.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Placed in Efficient reasoning and agent inference, one of 12 directions in Efficient LLM inference and serving. Builds on 29 of its 55 references that are in the map. First posted on arXiv 6 Sep 2026.

Builds on

29 of 55 references.

Closest ancestors · 1

Same direction, newest first

  1. 2025Runaway is Ashamed, But Helpful: On the Early-Exit Behavior of Large Language Model-based Agents in Embodied Environments23 May

Other references · 28

  1. 2024The Llama 3 Herd of Modelscited by 14,192
  2. 2023G-Eval: NLG Evaluation using GPT-4 with Better Human Alignmentcited by 2,205
  3. 2023FActScore: Fine-grained Atomic Evaluation of Factual Precision in Long Form Text Generationcited by 1,259
  4. 2023FrugalGPT: How to Use Large Language Models While Reducing Cost and Improving Performancecited by 791
  5. 2024WildChat: 1M ChatGPT Interaction Logs in the Wildcited by 598
  6. 2017BranchyNet: Fast Inference via Early Exiting from Deep Neural Networkscited by 591
  7. 2023Eliciting Latent Predictions from Transformers with the Tuned Lenscited by 579
  8. 2024Hybrid LLM: Cost-Efficient and Quality-Aware Query Routingcited by 354
  9. 2020DeeBERT: Dynamic Early Exiting for Accelerating BERT Inferencecited by 322
  10. 2020FastBERT: a Self-distilling BERT with Adaptive Inference Timecited by 224
  11. 2024Assisting in Writing Wikipedia-like Articles From Scratch with Large Language Modelscited by 150
  12. 2023Large Language Model Cascades with Mixture of Thoughts Representations for Cost-efficient Reasoningcited by 140
16 more
  1. 2023LongEval: Guidelines for Human Evaluation of Faithfulness in Long-form Summarizationcited by 96
  2. 2024MAgICoRe: Multi-Agent, Iterative, Coarse-to-Fine Refinement for Reasoningcited by 40
  3. 2025ConfTuner: Training Large Language Models to Express Their Confidence Verballycited by 37
  4. 2023WebBrain: Learning to Generate Factually Correct Articles for Queries by Grounding on Large Web Corpuscited by 29
  5. 2024WildHallucinations: Evaluating Long-form Factuality in LLMs with Real-World Entity Queriescited by 26
  6. 2025Leveraging Uncertainty Estimation for Efficient LLM Routingcited by 21
  7. 2025Conformal Linguistic Calibration: Trading-off between Factuality and Specificitycited by 20
  8. 2025Is That Your Final Answer? Test-Time Scaling Improves Selective Question Answeringcited by 17
  9. 2024Agreement-Based Cascading for Efficient Inferencecited by 16
  10. 2024Core: Robust Factual Precision with Informative Sub-Claim Identificationcited by 15
  11. 2023ConsistentEE: A Consistent and Hardness-Guided Early Exiting Method for Accelerating Language Models Inferencecited by 13
  12. 2023SmartBERT: A Promotion of Dynamic Early Exiting Mechanism for Accelerating BERT Inferencecited by 9
  13. 2025Always Tell Me The Odds: Fine-grained Conditional Probability Estimationcited by 9
  14. 2025VeriFastScore: Speeding up long-form factuality evaluationcited by 8
  15. 2025Semantic Agreement Enables Efficient Open-Ended LLM Cascadescited by 7
  16. 2025Logit-Entropy Adaptive Stopping Heuristic for Efficient Chain-of-Thought Reasoningcited by 1

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2026IDRBench: Benchmarking the Interactive Capabilities of Deep Research Agents
  2. 2026Mind2Report: Expert-Level Commercial Report Synthesis via Cognitive Deep Research Agent
  3. 2026Beyond Single-shot Writing: Deep Research Agents are Unreliable at Multi-turn Report Revision
  4. 2026DeepSynth-Eval: Objectively Evaluating Information Consolidation in Deep Survey Writing
  5. 2025ReportBench: Evaluating Deep Research Agents via Academic Survey Tasks
  6. 2026MiroEval: Benchmarking Multimodal Deep Research Agents in Process and Outcome
  7. 2025Deep Research: A Survey of Autonomous Research Agents
  8. 2026Deep-Reporter: Deep Research for Grounded Multimodal Long-Form Generation