# Interstellar: Using Halide's Scheduling Language to Analyze DNN Accelerators

arXiv: https://arxiv.org/abs/1809.04070 · HTML: https://mlsanity.com/paper/1809.04070/

## Key facts

- Cited by 34 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 10 Sep 2018.

## Built on by (most cited first, 30 of 34)

- [Full Stack Optimization of Transformer Inference: a Survey](https://mlsanity.com/paper/2302.14017/) (2023)
- [FLAT: An Optimized Dataflow for Mitigating Attention Bottlenecks](https://mlsanity.com/paper/2107.06419/) (2021)
- [CoSA: Scheduling by Constrained Optimization for Spatial Accelerators](https://mlsanity.com/paper/2105.01898/) (2021)
- [A Full-Stack Search Technique for Domain Optimized Deep Learning Accelerators](https://mlsanity.com/paper/2105.12842/) (2021)
- [A Hardware Evaluation Framework for Large Language Model Inference](https://mlsanity.com/paper/2312.03134/) (2023)
- [HASCO: Towards Agile HArdware and Software CO-design for Tensor Computation](https://mlsanity.com/paper/2105.01585/) (2021)
- [Performance Modeling and Workload Analysis of Distributed Large Language Model Training and Inference](https://mlsanity.com/paper/2407.14645/) (2024)
- [Enabling Resource-efficient AIoT System with Cross-level Optimization: A survey](https://mlsanity.com/paper/2309.15467/) (2023)
- [Union: A Unified HW-SW Co-Design Ecosystem in MLIR for Evaluating Tensor Operations on Spatial Accelerators](https://mlsanity.com/paper/2109.07419/) (2021)
- [SPADE: Sparse Pillar-based 3D Object Detection Accelerator for Autonomous Driving](https://mlsanity.com/paper/2305.07522/) (2023)
- [Understanding the Performance Horizon of the Latest ML Workloads with NonGEMM Workloads](https://mlsanity.com/paper/2404.11788/) (2024)
- [DOSA: Differentiable Model-Based One-Loop Search for DNN Accelerators](https://mlsanity.com/paper/2509.10702/) (2025)
- [Design of High-Throughput Mixed-Precision CNN Accelerators on FPGA](https://mlsanity.com/paper/2208.04854/) (2022)
- [BOLD: Boolean Logic Deep Learning](https://mlsanity.com/paper/2405.16339/) (2024)
- [SystolicAttention: Fusing FlashAttention within a Single Systolic Array](https://mlsanity.com/paper/2507.11331/) (2025)
- [Evaluating Spatial Accelerator Architectures with Tiled Matrix-Matrix Multiplication](https://mlsanity.com/paper/2106.10499/) (2021)
- [Demystifying Map Space Exploration for NPUs](https://mlsanity.com/paper/2210.03731/) (2022)
- [SALSA: Simulated Annealing based Loop-Ordering Scheduler for DNN Accelerators](https://mlsanity.com/paper/2304.12931/) (2023)
- [Boolean Variation and Boolean Logic BackPropagation](https://mlsanity.com/paper/2311.07427/) (2023)
- [Vision Transformer Computation and Resilience for Dynamic Inference](https://mlsanity.com/paper/2212.02687/) (2022)
- [YFlows: Systematic Dataflow Exploration and Code Generation for Efficient Neural Network Inference using SIMD Architectures on CPUs](https://mlsanity.com/paper/2310.00574/) (2023)
- [PACiM: A Sparsity-Centric Hybrid Compute-in-Memory Architecture via Probabilistic Approximation](https://mlsanity.com/paper/2408.16246/) (2024)
- [SmartDeal: Re-Modeling Deep Network Weights for Efficient Inference and Training](https://mlsanity.com/paper/2101.01163/) (2021)
- [Optimizing the Whole-life Cost in End-to-end CNN Acceleration](https://mlsanity.com/paper/2104.05541/) (2021)
- [Compiler-Driven Simulation of Reconfigurable Hardware Accelerators](https://mlsanity.com/paper/2202.00739/) (2022)
- [Shisha: Online scheduling of CNN pipelines on heterogeneous architectures](https://mlsanity.com/paper/2202.11575/) (2022)
- [DRAGON (Differentiable Graph Execution) : A suite of Hardware Simulation and Optimization tools for Modern AI/Non-AI Workloads](https://mlsanity.com/paper/2204.06676/) (2022)
- [KAPLA: Pragmatic Representation and Fast Solving of Scalable NN Accelerator Dataflow](https://mlsanity.com/paper/2306.15676/) (2023)
- [Performance Analysis of DNN Inference/Training with Convolution and non-Convolution Operations](https://mlsanity.com/paper/2306.16767/) (2023)
- [A Survey on Design Methodologies for Accelerating Deep Learning on Heterogeneous Architectures](https://mlsanity.com/paper/2311.17815/) (2023)

## Abstract (arXiv)

We show that DNN accelerator micro-architectures and their program mappings represent specific choices of loop order and hardware parallelism for computing the seven nested loops of DNNs, which enables us to create a formal taxonomy of all existing dense DNN accelerators. Surprisingly, the loop transformations needed to create these hardware variants can be precisely and concisely represented by Halide's scheduling language. By modifying the Halide compiler to generate hardware, we create a system that can fairly compare these prior accelerators. As long as proper loop blocking schemes are used, and the hardware can support mapping replicated loops, many different hardware dataflows yield similar energy efficiency with good performance. This is because the loop blocking can ensure that most data references stay on-chip with good locality and the processing units have high resource utilization. How resources are allocated, especially in the memory system, has a large impact on energy and performance. By optimizing hardware resource allocation while keeping throughput constant, we achieve up to 4.2X energy improvement for Convolutional Neural Networks (CNNs), 1.6X and 1.8X improvement for Long Short-Term Memories (LSTMs) and multi-layer perceptrons (MLPs), respectively.