Auto-Differentiation of Relational Computations for Very Large Scale Machine Learning
Not on the map. Why?
Abstract (arXiv)
The relational data model was designed to facilitate large-scale data management and analytics. We consider the problem of how to differentiate computations expressed relationally. We show experimentally that a relational engine running an auto-differentiated relational algorithm can easily scale to very large datasets, and is competitive with state-of-the-art, special-purpose systems for large-scale distributed machine learning.
Key facts. Cited by 1 paper in the map (data through 14 Oct 2026). Builds on 45 of its 94 references that are in the map. First posted on arXiv 31 May 2023.
Builds on
45 of 94 references.
References · 45
- 2016Semi-Supervised Classification with Graph Convolutional Networkscited by 11,361
- 2019Megatron-LM: Training Multi-Billion Parameter Language Models Using Model Parallelismcited by 1,868
- 2020GShard: Scaling Giant Models with Conditional Computation and Automatic Shardingcited by 1,631
- 2015Automatic differentiation in machine learning: a surveycited by 855
- 2020Knowledge Graphscited by 621
- 2019DiffTaichi: Differentiable Programming for Physical Simulationcited by 258
- 2015MXNet: A Flexible and Efficient Machine Learning Library for Heterogeneous Distributed Systemscited by 195
- 2018Beyond Data and Model Parallelism for Deep Neural Networkscited by 179
- 2018Mesh-TensorFlow: Deep Learning for Supercomputerscited by 169
- 2019AliGraph: A Comprehensive Graph Neural Network Platformcited by 153
- 2020DistDGL: Distributed Graph Neural Network Training for Billion-Scale Graphscited by 139
- 2020Towards Deeper Graph Neural Networks with Differentiable Group Normalizationcited by 114
33 more
- 2021GSPMD: General and Scalable Parallelization for ML Computation Graphscited by 111
- 2016A Differentiable Physics Engine for Deep Learning in Roboticscited by 108
- 2020NeuralSim: Augmenting Differentiable Simulators with Neural Networkscited by 82
- 2022Pathways: Asynchronous Distributed Dataflow for MLcited by 82
- 2020DGL-KE: Training Knowledge Graph Embeddings at Scalecited by 67
- 2020A mathematical model for automatic differentiation in machine learningcited by 45
- 2022PipeGCN: Efficient Full-Graph Training of Graph Convolutional Networks with Pipelined Feature Communicationcited by 45
- 2022BNS-GCN: Efficient Full-Graph Training of Graph Convolutional Networks with Partition-Parallelism and Random Boundary Node Samplingcited by 43
- 2022Automatic Differentiation of Programs with Discrete Randomnesscited by 29
- 2022Automatic differentiation of nonsmooth iterative algorithmscited by 25
- 2020Super-efficiency of automatic differentiation for functions defined as a minimumcited by 24
- 2020Randomized Automatic Differentiationcited by 23
- 2019Chainer: A Deep Learning Framework for Accelerating the Research Cyclecited by 22
- 2018Automatic differentiation in ML: Where we are and where we should be goingcited by 20
- 2020On Correctness of Automatic Differentiation for Non-Differentiable Functionscited by 20
- 2022End-to-end Optimization of Machine Learning Prediction Queriescited by 16
- 2022Serving Deep Learning Models with Deduplication from Relational Databasescited by 15
- 2022Query Processing on Tensor Computation Runtimescited by 12
- 2019A Simple Differentiable Programming Languagecited by 11
- 2021Storchastic: A Framework for General Stochastic Automatic Differentiationcited by 11
- 2022Scalable First-Order Bayesian Optimization via Structured Automatic Differentiationcited by 10
- 2020Augmenting Differentiable Simulators with Neural Networks to Close the Sim2Real Gapcited by 9
- 2018Efficient Differentiable Programming in a Functional Array-Processing Languagecited by 8
- 2017Auto-Differentiating Linear Algebracited by 7
- 2022Share the Tensor Tea: How Databases can Leverage the Machine Learning Ecosystemcited by 7
- 2017PlinyCompute: A Platform for High-Performance, Distributed, Data-Intensive Tool Developmentcited by 5
- 2017Freebase-triples: A Methodology for Processing the Freebase Data Dumpscited by 5
- 2020Relational Algorithms for k-means Clusteringcited by 4
- 2021In-Database Regression in Input Sparsity Timecited by 3
- 2023A Comparison of Decision Forest Inference Platforms from A Database Perspectivecited by 3
- 2018Tangent: Automatic differentiation using source-code transformation for dynamically typed array programmingcited by 2
- 2018Learning in Integer Latent Variable Models with Nested Automatic Differentiationcited by 1
- 2019tspDB: Time Series Predict DBcited by 1
Built on by
Citing papers per year: 2023: 1.
1 paper.