# DyNet: The Dynamic Neural Network Toolkit

arXiv: https://arxiv.org/abs/1701.03980 · HTML: https://mlsanity.com/paper/1701.03980/

## Key facts

- Cited by 21 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 15 Jan 2017.

## Built on by (most cited first, 21 of 21)

- [MasakhaNER: Named Entity Recognition for African Languages](https://mlsanity.com/paper/2103.11811/) (2021)
- [Torch.fx: Practical Program Capture and Transformation for Deep Learning in Python](https://mlsanity.com/paper/2112.08429/) (2021)
- [Lexically Aware Semi-Supervised Learning for OCR Post-Correction](https://mlsanity.com/paper/2111.02622/) (2021)
- [LazyTensor: combining eager execution with domain-specific compilers](https://mlsanity.com/paper/2102.13267/) (2021)
- [Swift for TensorFlow: A portable, flexible platform for deep learning](https://mlsanity.com/paper/2102.13243/) (2021)
- [VibeTensor: System Software for Deep Learning, Fully Generated by AI Agents](https://mlsanity.com/paper/2601.16238/) (2026)
- [Optimizing Mixture of Experts using Dynamic Recompilations](https://mlsanity.com/paper/2205.01848/) (2022)
- [To Build Our Future, We Must Know Our Past: Contextualizing Paradigm Shifts in Natural Language Processing](https://mlsanity.com/paper/2310.07715/) (2023)
- [SoD2: Statically Optimizing Dynamic Deep Neural Network](https://mlsanity.com/paper/2403.00176/) (2024)
- [Yggdrasil: Bridging Dynamic Speculation and Static Runtime for Latency-Optimal Tree-Based LLM Decoding](https://mlsanity.com/paper/2512.23858/) (2025)
- [Boosting Convolutional Neural Networks with Middle Spectrum Grouped Convolution](https://mlsanity.com/paper/2304.06305/) (2023)
- [Adaptive and Resource-efficient Agentic AI Systems for Mobile and Embedded Devices: A Survey](https://mlsanity.com/paper/2510.00078/) (2025)
- [This Must Be the Place: Predicting Engagement of Online Communities in a Large-scale Distributed Campaign](https://mlsanity.com/paper/2201.05334/) (2022)
- [Terra: Imperative-Symbolic Co-Execution of Imperative Deep Learning Programs](https://mlsanity.com/paper/2201.09210/) (2022)
- [ED-Batch: Efficient Automatic Batching of Dynamic Neural Networks via Learned Finite State Machines](https://mlsanity.com/paper/2302.03851/) (2023)
- [ACRoBat: Optimizing Auto-batching of Dynamic Deep Learning at Compile Time](https://mlsanity.com/paper/2305.10611/) (2023)
- [Learning Neural Models for Natural Language Processing in the Face of Distributional Shift](https://mlsanity.com/paper/2109.01558/) (2021)
- [Monadic Deep Learning](https://mlsanity.com/paper/2307.12187/) (2023)
- [Recursive Function Definitions in Static Dataflow Graphs and their Implementation in TensorFlow](https://mlsanity.com/paper/2410.20225/) (2024)
- [Neuroplasticity-inspired dynamic ANNs for multi-task demand forecasting](https://mlsanity.com/paper/2509.24495/) (2025)
- [TASTE: Throughput-Aware Batch Size Tuning for On-Device Edge Learning](https://mlsanity.com/paper/2609.07444/) (2026)

## Abstract (arXiv)

We describe DyNet, a toolkit for implementing neural network models based on dynamic declaration of network structure. In the static declaration strategy that is used in toolkits like Theano, CNTK, and TensorFlow, the user first defines a computation graph (a symbolic representation of the computation), and then examples are fed into an engine that executes this computation and computes its derivatives. In DyNet's dynamic declaration strategy, computation graph construction is mostly transparent, being implicitly constructed by executing procedural code that computes the network outputs, and the user is free to use different network structures for each input. Dynamic declaration thus facilitates the implementation of more complicated network architectures, and DyNet is specifically designed to allow users to implement their models in a way that is idiomatic in their preferred programming language (C++ or Python). One challenge with dynamic declaration is that because the symbolic computation graph is defined anew for every training example, its construction must have low overhead. To achieve this, DyNet has an optimized C++ backend and lightweight graph representation. Experiments show that DyNet's speeds are faster than or comparable with static declaration toolkits, and significantly faster than Chainer, another dynamic declaration toolkit. DyNet is released open-source under the Apache 2.0 license and available at http://github.com/clab/dynet.