mlsanityData to 14 Oct 2026

Operator Fusion in XLA: Analysis and Evaluation

arXiv 2301.13062 ↗First posted 30 Jan 2023cs.LGcited by 9

Not on the map. Why?

Abstract (arXiv)

Machine learning (ML) compilers are an active area of research because they offer the potential to automatically speedup tensor programs. Kernel fusion is often cited as an important optimization performed by ML compilers. However, there exists a knowledge gap about how XLA, the most common ML compiler, applies this nuanced optimization, what kind of speedup it can afford, and what low-level effects it has on hardware. Our paper aims to bridge this knowledge gap by studying key compiler passes of XLA's source code. Our evaluation on a reinforcement learning environment Cartpole shows how different fusion decisions in XLA are made in practice. Furthermore, we implement several XLA kernel fusion strategies that can achieve up to 10.56x speedup compared to our baseline implementation.

Key facts. Cited by 9 papers in the map (data through 14 Oct 2026). Builds on 12 of its 20 references that are in the map. First posted on arXiv 30 Jan 2023.

Builds on

12 of 20 references.

References · 12

  1. 2017Attention Is All You Needcited by 60,361
  2. 2019PyTorch: An Imperative Style, High-Performance Deep Learning Librarycited by 20,305
  3. 2020GShard: Scaling Giant Models with Conditional Computation and Automatic Shardingcited by 1,631
  4. 2021Brax -- A Differentiable Physics Engine for Large Scale Rigid Body Simulationcited by 407
  5. 2020Ansor: Generating High-Performance Tensor Programs for Deep Learningcited by 201
  6. 2015MXNet: A Flexible and Efficient Machine Learning Library for Heterogeneous Distributed Systemscited by 195
  7. 2020Data Movement Is All You Need: A Case Study on Optimizing Transformerscited by 108
  8. 2018Tensor Comprehensions: Framework-Agnostic High-Performance Machine Learning Abstractionscited by 85
  9. 2018TVM: An Automated End-to-End Optimizing Compiler for Deep Learningcited by 62
  10. 2021DNNFusion: Accelerating Deep Neural Networks Execution with Advanced Operator Fusioncited by 57
  11. 2020The Deep Learning Compiler: A Comprehensive Surveycited by 46
  12. 2021Horizontally Fused Training Array: An Effective Hardware Utilization Squeezer for Training Novel Deep Learning Modelscited by 7

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2023: 1; 2024: 2; 2025: 1; 2026: 5.

9 papers.

  1. 2025QiMeng-Kernel: Macro-Thinking Micro-Coding Paradigm for LLM-Based High-Performance GPU Kernel Generationcited by 19
  2. 2026Faster Predictive Coding Networks via Better Initializationcited by 3
  3. 2024FastCHGNet: Training one Universal Interatomic Potential to 1.5 Hours with 32 GPUscited by 2
  4. 2023Survey on Computer Vision Techniques for Internet-of-Things Devicescited by 0
  5. 2024Graph neural networks with configuration cross-attention for tensor compilerscited by 0
  6. 2026Step-TP: A Grounded, Step-Level Dataset with Chain-of-Thought Reasoning for LLM-Guided Tensor Program Optimizationcited by 0
  7. 2026Nova: An End-to-End MLIR Compiler for Deep Learningcited by 0
  8. 2026Measuring and Reducing WebGPU Dispatch Overhead for LLM Inferencecited by 0
  9. 2026KernelZero: Co-Evolving Proposer and Coder for Continuously Improved GPU Kernel Generationcited by 0