# On Optimizing Operator Fusion Plans for Large-Scale Machine Learning in SystemML

arXiv: https://arxiv.org/abs/1801.00829 · HTML: https://mlsanity.com/paper/1801.00829/

## Key facts

- Cited by 13 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 2 Jan 2018.

## Built on by (most cited first, 13 of 13)

- [DNNFusion: Accelerating Deep Neural Networks Execution with Advanced Operator Fusion](https://mlsanity.com/paper/2108.13342/) (2021)
- [Achieving on-Mobile Real-Time Super-Resolution with Neural Architecture and Pruning Search](https://mlsanity.com/paper/2108.08910/) (2021)
- [A Full-Stack Search Technique for Domain Optimized Deep Learning Accelerators](https://mlsanity.com/paper/2105.12842/) (2021)
- [Enabling Resource-efficient AIoT System with Cross-level Optimization: A survey](https://mlsanity.com/paper/2309.15467/) (2023)
- [Automatic Mapping of the Best-Suited DNN Pruning Schemes for Real-Time Mobile Acceleration](https://mlsanity.com/paper/2111.11581/) (2021)
- [TOD: GPU-accelerated Outlier Detection via Tensor Operations](https://mlsanity.com/paper/2110.14007/) (2021)
- [A Compression-Compilation Framework for On-mobile Real-time BERT Applications](https://mlsanity.com/paper/2106.00526/) (2021)
- [Collage: Seamless Integration of Deep Learning Backends with Automatic Placement](https://mlsanity.com/paper/2111.00655/) (2021)
- [Saturn: An Optimized Data System for Large Model Deep Learning Workloads](https://mlsanity.com/paper/2309.01226/) (2023)
- [REDS: Resource-Efficient Deep Subnetworks for Dynamic Resource Constraints](https://mlsanity.com/paper/2311.13349/) (2023)
- [The Duck's Brain: Training and Inference of Neural Networks in Modern Database Engines](https://mlsanity.com/paper/2312.17355/) (2023)
- [Moirai: Towards Optimal Placement for Distributed Inference on Heterogeneous Devices](https://mlsanity.com/paper/2312.04025/) (2023)
- [stratum: A System Infrastructure for Massive Agent-Centric ML Workloads](https://mlsanity.com/paper/2603.03589/) (2026)

## Abstract (arXiv)

Many large-scale machine learning (ML) systems allow specifying custom ML algorithms by means of linear algebra programs, and then automatically generate efficient execution plans. In this context, optimization opportunities for fused operators---in terms of fused chains of basic operators---are ubiquitous. These opportunities include (1) fewer materialized intermediates, (2) fewer scans of input data, and (3) the exploitation of sparsity across chains of operators. Automatic operator fusion eliminates the need for hand-written fused operators and significantly improves performance for complex or previously unseen chains of operations. However, existing fusion heuristics struggle to find good fusion plans for complex DAGs or hybrid plans of local and distributed operations. In this paper, we introduce an optimization framework for systematically reason about fusion plans that considers materialization points in DAGs, sparsity exploitation, different fusion template types, as well as local and distributed operations. In detail, we contribute algorithms for (1) candidate exploration of valid fusion plans, (2) cost-based candidate selection, and (3) code generation of local and distributed operations over dense, sparse, and compressed data. Our experiments in SystemML show end-to-end performance improvements with optimized fusion plans of up to 21x compared to hand-written fused operators, with negligible optimization and code generation overhead.