mlsanityData to 14 Oct 2026

Trident: Unifying Guarded Dispatch and Host Execution for PyTorch Triton Workloads

arXiv 2609.37241 ↗First posted 29 Sep 2026cs.LGcited by 0

Not on the map. Why?

Abstract (arXiv)

User-written Triton kernels enable high-performance GPU computation within PyTorch, but their end-to-end latency can remain dominated by host-side orchestration, especially when device execution is short. Although torch.compile can generate native host wrappers for captured graphs, each invocation still passes through runtime-managed specialization lookup, guard evaluation, and preparation before reaching the wrapper. We present Trident, a compiler backend that removes this recurring overhead from the specialization cache-hit path. Trident introduces the Specialization Cache Module (SCM), which compiles guarded specialization selection, argument and execution-environment preparation, and host execution for multiple specializations into a single executable module. An invocation enters the SCM once, remains in compiled code when a specialization matches, and returns to Python only when a new specialization must be compiled. Built on Torch-MLIR, Trident lowers guards and host-side orchestration to native code while retaining calls to optimized runtime implementations of supported ATen operators. Our evalu- ation on two LLMs shows that Trident achieves up to a 1.47x speedup in model-level end-to-end latency over eager execution and up to 1.68x over torch.compile.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 14 of its 33 references that are in the map. First posted on arXiv 29 Sep 2026.

Builds on

14 of 33 references.

References · 14

  1. 2019PyTorch: An Imperative Style, High-Performance Deep Learning Librarycited by 20,305
  2. 2023GPT-4 Technical Reportcited by 18,396
  3. 2021Training Verifiers to Solve Math Word Problemscited by 10,109
  4. 2025Qwen3 Technical Reportcited by 8,820
  5. 2020Measuring Massive Multitask Language Understandingcited by 7,963
  6. 2023Efficient Memory Management for Large Language Model Serving with PagedAttentioncited by 5,944
  7. 2023SGLang: Efficient Execution of Structured Language Model Programscited by 1,141
  8. 2026DeepSeek-V4: Towards Highly Efficient Million-Token Context Intelligencecited by 1,040
  9. 2014Caffe: Convolutional Architecture for Fast Feature Embeddingcited by 288
  10. 2016Theano: A Python framework for fast computation of mathematical expressionscited by 64
  11. 2021Torch.fx: Practical Program Capture and Transformation for Deep Learning in Pythoncited by 40
  12. 2025MPK: A Compiler and Runtime for Mega-Kernelizing Tensor Programscited by 15
2 more
  1. 2025FlashMoE: Fast Distributed MoE in a Single Kernelcited by 14
  2. 2018JANUS: Fast and Flexible Deep Learning via Symbolic Graph Execution of Imperative Programscited by 4

Built on by

Not cited yet.