mlsanityData to 14 Oct 2026

FastTTS: Accelerating Test-Time Scaling for Edge LLM Reasoning

arXiv 2509.00195 ↗First posted 29 Aug 2025cs.LGcited by 3

On the map: LLM reasoning → Reasoning for agents, planning and code

also in 1 more
Abstract (arXiv)

Recent advances in reasoning Large Language Models (LLMs) are driving the emergence of agentic AI systems. Edge deployment of LLM agents near end users is increasingly necessary to protect data privacy, enable offline use, and provide responsive interaction with local context. However, strict memory constraints on edge devices limit deployment to smaller LLMs, whose reasoning capabilities are much weaker than those of large cloud models, hindering practical deployment of edge agentic AI. Test-Time Scaling (TTS) offers a promising solution by allocating more compute during inference to enhance the reasoning capability of edge LLMs. However, current TTS methods introduce heavy hardware performance overhead on resource-constrained devices, making them impractical for real applications. To address this challenge, we present FastTTS, a serving system that enables fast and efficient TTS for memory-constrained LLM reasoning. After analyzing common patterns across various TTS methods and identifying their performance bottlenecks, we introduce three novel techniques: i) Speculative Beam Extension, which mitigates system stragglers caused by irregular reasoning paths, ii) Asymmetric Multi-Model Memory Allocation, which dynamically balances memory usage between token generation and reasoning-step verification, and iii) Dynamic Prefix-Aware Scheduling, which optimizes reasoning execution to maximize KV-cache reuse across search paths. FastTTS offers a plug-and-play third-party library on top of vLLM, enabling edge LLMs on a single consumer GPU to match cloud-model accuracy and cloud-measured latency. Comprehensive evaluation shows that FastTTS achieves an average 2.2x higher goodput and reduces latency by 38%--68% compared to the vLLM baseline; it pushes the boundaries of low-latency TTS on memory-constrained edge devices and highlights the potential for democratizing agentic AI.

Key facts. Cited by 3 papers in the map (data through 14 Oct 2026). Placed in Reasoning for agents, planning and code, one of 17 directions in LLM reasoning. Builds on 44 of its 51 references that are in the map. First posted on arXiv 29 Aug 2025.

Builds on

44 of 51 references.

References · 44

  1. 2021Training Verifiers to Solve Math Word Problemscited by 10,109
  2. 2024DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Modelscited by 9,379
  3. 2025Qwen3 Technical Reportcited by 8,820
  4. 2023Efficient Memory Management for Large Language Model Serving with PagedAttentioncited by 5,944
  5. 2023Let's Verify Step by Stepcited by 3,789
  6. 2024Scaling LLM Test-Time Compute Optimally can be More Effective than Scaling Model Parameterscited by 1,986
  7. 2024DeepSeek-Coder: When the Large Language Model Meets Programming -- The Rise of Code Intelligencecited by 1,181
  8. 2023SGLang: Efficient Execution of Structured Language Model Programscited by 1,141
  9. 2024Qwen2.5-Math Technical Report: Toward Mathematical Expert Model via Self-Improvementcited by 1,090
  10. 2023Math-Shepherd: Verify and Reinforce LLMs Step-by-step without Human Annotationscited by 928
  11. 2024Medusa: Simple LLM Inference Acceleration Framework with Multiple Decoding Headscited by 843
  12. 2024DistServe: Disaggregating Prefill and Decoding for Goodput-optimized Large Language Model Servingcited by 617
32 more
  1. 2022DeepSpeed Inference: Enabling Efficient Inference of Transformer Models at Unprecedented Scalecited by 407
  2. 2024EAGLE-2: Faster Inference of Language Models with Dynamic Draft Treescited by 370
  3. 2025EAGLE-3: Scaling up Inference Acceleration of Large Language Models via Training-Time Testcited by 331
  4. 2024Break the Sequential Dependency of LLM Inference Using Lookahead Decodingcited by 298
  5. 2023Alphazero-like Tree-Search can Guide Large Language Model Decoding and Trainingcited by 295
  6. 2024InfiniGen: Efficient Generative Inference of Large Language Models with Dynamic KV Cache Managementcited by 245
  7. 2025A Survey on Test-Time Scaling in Large Language Models: What, How, Where, and How Well?cited by 223
  8. 2023PowerInfer: Fast Large Language Model Serving with a Consumer-grade GPUcited by 217
  9. 2023LLM in a flash: Efficient Large Language Model Inference with Limited Memorycited by 177
  10. 2024Llumnix: Dynamic Scheduling for Large Language Model Servingcited by 174
  11. 2025Can 1B LLM Surpass 405B LLM? Rethinking Compute-Optimal Test-Time Scalingcited by 136
  12. 2024LoongServe: Efficiently Serving Long-Context Large Language Models with Elastic Sequence Parallelismcited by 112
  13. 2025Reward-Guided Speculative Decoding for Efficient LLM Reasoningcited by 106
  14. 2024A Review on Edge Large Language Models: Design, Execution, and Applicationscited by 100
  15. 2024Parrot: Efficient Serving of LLM-based Applications with Semantic Variablecited by 99
  16. 2025GenPRM: Scaling Test-Time Compute of Process Reward Models via Generative Reasoningcited by 97
  17. 2025SpecReason: Fast and Accurate Inference-Time Compute via Speculative Reasoningcited by 59
  18. 2024Efficiently Scaling LLM Reasoning with Certaindexcited by 58
  19. 2024FastDecode: High-Throughput GPU-Efficient LLM Serving using Heterogeneous Pipelinescited by 49
  20. 2024BatchLLM: Optimizing Large Batched LLM Inference with Global Prefix Sharing and Throughput-oriented Token Batchingcited by 32
  21. 2024EDGE-LLM: Enabling Efficient Large Language Model Adaptation on Edge Devices via Layerwise Unified Compression and Adaptive Layer Tuning and Votingcited by 27
  22. 2025Speculative Decoding and Beyond: An In-Depth Survey of Techniquescited by 23
  23. 2024Accelerating Retrieval-Augmented Language Model Serving with Speculationcited by 20
  24. 2024Hardware-Aware Parallel Prompt Decoding for Memory-Efficient Acceleration of LLM Inferencecited by 16
  25. 2025ETS: Efficient Tree Search for Inference-Time Scalingcited by 16
  26. 2025Device-Cloud Collaborative LLM Inference with Multi-Modal, Multi-Task, Multi-Turn Conversationscited by 14
  27. 2025Oaken: Fast and Efficient LLM Serving with Online-Offline Hybrid KV Cache Quantizationcited by 14
  28. 2025Scaling Speculative Decoding with Lookahead Reasoningcited by 13
  29. 2025Agent.xpu: Efficient Scheduling of Agentic LLM Workloads on Heterogeneous SoCcited by 11
  30. 2025Exploring Code Language Models for Automated HLS-based Hardware Generation: Benchmark, Infrastructure and Analysiscited by 10
  31. 2025Rethinking Optimal Verification Granularity for Compute-Efficient Test-Time Scalingcited by 5
  32. 2025HedraRAG: Coordinating LLM Generation and Database Retrieval in Heterogeneous RAG Servingcited by 3

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 3.

3 papers.

  1. 2026Type-Checked Compliance: Deterministic Guardrails for Agentic Financial Systems Using Lean 4 Theorem Provingcited by 2
  2. 2026Interpretable Adaptive Sampling for LLM Test-Time Scalingcited by 1
  3. 2026Hive: A Multi-Agent Infrastructure for Algorithm- and Task-Level Scalingcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2026LLMs Improving LLMs: Agentic Discovery for Test-Time Scaling
  2. 2025Reasoning on a Budget: A Survey of Adaptive and Controllable Test-Time Compute in LLMs
  3. 2026End-to-End Latency-Minimizing and Load-Balanced Request Scheduling for Edge LLM Inference in Agentic AI Services
  4. 2025Scaling Test-time Compute for LLM Agents
  5. 2025Are We Scaling the Right Thing? A System Perspective on Test-Time Scaling
  6. 2025A Survey of Scaling in Large Language Model Reasoning
  7. 2025Resource-Aware LLM Reasoning for Mobile Edge General Intelligence
  8. 2026Efficient Reasoning on the Edge