mlsanityData to 14 Oct 2026

TimelyLLM: Segmented LLM Serving System for Time-sensitive Robotic Applications

arXiv 2412.18695 ↗First posted 24 Dec 2024cs.RO cs.DC cs.LGcited by 1

Not on the map. Why?

Abstract (arXiv)

Large Language Models (LLMs) such as GPT-4 and Llama3 can already comprehend complex commands and process diverse tasks. This advancement facilitates their application in controlling drones and robots for various tasks. However, existing LLM serving systems typically employ a first-come, first-served (FCFS) batching mechanism, which fails to address the time-sensitive requirements of robotic applications. To address it, this paper proposes a new system named TimelyLLM serving multiple robotic agents with time-sensitive requests. TimelyLLM introduces novel mechanisms of segmented generation and scheduling that optimally leverage redundancy between robot plan generation and execution phases. We report an implementation of TimelyLLM on a widely-used LLM serving framework and evaluate it on a range of robotic applications. Our evaluation shows that TimelyLLM improves the time utility up to 1.97x, and reduces the overall waiting time by 84%.

Key facts. Cited by 1 paper in the map (data through 14 Oct 2026). Builds on 18 of its 61 references that are in the map. First posted on arXiv 24 Dec 2024.

Builds on

18 of 61 references.

References · 18

  1. 2020Language Models are Few-Shot Learnerscited by 32,419
  2. 2023GPT-4 Technical Reportcited by 18,396
  3. 2024GPT-4o System Cardcited by 4,588
  4. 2024Phi-3 Technical Report: A Highly Capable Language Model Locally on Your Phonecited by 1,917
  5. 2018Hierarchical Neural Story Generationcited by 1,193
  6. 2023Robots That Ask For Help: Uncertainty Alignment for Large Language Model Plannerscited by 308
  7. 2023SayPlan: Grounding Large Language Models using 3D Scene Graphs for Scalable Robot Task Planningcited by 294
  8. 2023An LLM Compiler for Parallel Function Callingcited by 179
  9. 2023REFLECT: Summarizing Robot Experiences for Failure Explanation and Correctioncited by 175
  10. 2023Fast Distributed Inference Serving for Large Language Modelscited by 145
  11. 2023Bootstrap Your Own Skills: Learning to Solve New Tasks with Large Language Model Guidancecited by 86
  12. 2024Efficient LLM Scheduling by Learning to Rankcited by 85
6 more
  1. 2024Efficient Interactive LLM Serving with Proxy Model-based Sequence Length Predictioncited by 66
  2. 2024LLM3:Large Language Model-based Task and Motion Planning with Motion Failure Reasoningcited by 63
  3. 2023Errors are Useful Prompts: Instruction Guided Task Programming with Verifier-Assisted Iterative Promptingcited by 46
  4. 2024Queue management for slo-oriented large language model servingcited by 36
  5. 2024Don't Stop Me Now: Embedding Based Scheduling for LLMscited by 35
  6. 2023TypeFly: Flying Drones with Large Language Modelcited by 23

Built on by

Cited by year

Citing papers per year: 2025: 1.

1 paper.

  1. 2025Streaming, Fast and Slow: Cognitive Load-Aware Streaming for Efficient LLM Servingcited by 3

Similar, not cited

Close in wording, no citation either way

  1. 2025Pretrained LLMs as Real-Time Controllers for Robot Operated Serial Production Line
  2. 2026When Do LLM Agents Help? Deadline-Aware Mixed-Criticality Task Scheduling at the Autonomous-Vehicle Edge
  3. 2026Hybrid LLM-based Intelligent Framework for Robot Task Scheduling
  4. 2023SMART-LLM: Smart Multi-Agent Robot Task Planning using Large Language Models
  5. 2025AuDeRe: Automated Strategy Decision and Realization in Robot Planning and Control via LLMs
  6. 2025VerifyLLM: LLM-Based Pre-Execution Task Plan Verification for Robots
  7. 2025Large language model-based task planning for service robots: A review
  8. 2025Large Language Models for Multi-Robot Systems: A Survey