# Effective Extensible Programming: Unleashing Julia on GPUs

arXiv: https://arxiv.org/abs/1712.03112 · HTML: https://mlsanity.com/paper/1712.03112/

## Key facts

- Cited by 16 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 8 Dec 2017.

## Built on by (most cited first, 16 of 16)

- [Comparing Llama-2 and GPT-3 LLMs for HPC kernels generation](https://mlsanity.com/paper/2309.07103/) (2023)
- [Evaluation of OpenAI Codex for HPC Parallel Programming Models Kernel Generation](https://mlsanity.com/paper/2306.15121/) (2023)
- [Branching Flows: Discrete, Continuous, and Manifold Flow Matching with Splits and Deletions](https://mlsanity.com/paper/2511.09465/) (2025)
- [Provably Scalable Black-Box Variational Inference with Structured Variational Families](https://mlsanity.com/paper/2401.10989/) (2024)
- [Generalizing Adam to Manifolds for Efficiently Training Transformers](https://mlsanity.com/paper/2305.16901/) (2023)
- [Volume-Preserving Transformers for Learning Time Series Data with Structure](https://mlsanity.com/paper/2312.11166/) (2023)
- [AbstractDifferentiation.jl: Backend-Agnostic Differentiable Programming in Julia](https://mlsanity.com/paper/2109.12449/) (2021)
- [GPU-Accelerated Forward-Backward algorithm with Application to Lattice-Free MMI](https://mlsanity.com/paper/2112.00709/) (2021)
- [Machines of finite depth: towards a formalization of neural networks](https://mlsanity.com/paper/2204.12786/) (2022)
- [Training neural network ensembles via trajectory sampling](https://mlsanity.com/paper/2209.11116/) (2022)
- [Dirac--Bianconi Graph Neural Networks -- Enabling Non-Diffusive Long-Range Graph Predictions](https://mlsanity.com/paper/2407.12419/) (2024)
- [EBIC.JL -- an Efficient Implementation of Evolutionary Biclustering Algorithm in Julia](https://mlsanity.com/paper/2105.01196/) (2021)
- [A High Throughput Generative Vector Autoregression Model for Stochastic Synapses](https://mlsanity.com/paper/2205.05053/) (2022)
- [Minibatch training of neural network ensembles via trajectory sampling](https://mlsanity.com/paper/2306.13442/) (2023)
- [GraphNeuralNetworks.jl: Deep Learning on Graphs with Julia](https://mlsanity.com/paper/2412.06354/) (2024)
- [JLIR: A Julia-Native MLIR-Inspired Intermediate Representation with Automatic JACC Kernel Extraction](https://mlsanity.com/paper/2609.04585/) (2026)

## Abstract (arXiv)

GPUs and other accelerators are popular devices for accelerating compute-intensive, parallelizable applications. However, programming these devices is a difficult task. Writing efficient device code is challenging, and is typically done in a low-level programming language. High-level languages are rarely supported, or do not integrate with the rest of the high-level language ecosystem. To overcome this, we propose compiler infrastructure to efficiently add support for new hardware or environments to an existing programming language. We evaluate our approach by adding support for NVIDIA GPUs to the Julia programming language. By integrating with the existing compiler, we significantly lower the cost to implement and maintain the new compiler, and facilitate reuse of existing application code. Moreover, use of the high-level Julia programming language enables new and dynamic approaches for GPU programming. This greatly improves programmer productivity, while maintaining application performance similar to that of the official NVIDIA CUDA toolkit.