# Pruning In Time (PIT): A Lightweight Network Architecture Optimizer for Temporal Convolutional Networks

arXiv: https://arxiv.org/abs/2203.14768 · HTML: https://mlsanity.com/paper/2203.14768/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- Builds on 12 of its 20 references that are in the map.
- First posted on arXiv 28 Mar 2022.

## Builds on (most cited first, 12 of 12)

- [DARTS: Differentiable Architecture Search](https://mlsanity.com/paper/1806.09055/) (2018)
- [Neural Architecture Search with Reinforcement Learning](https://mlsanity.com/paper/1611.01578/) (2016)
- [An Empirical Evaluation of Generic Convolutional and Recurrent Networks for Sequence Modeling](https://mlsanity.com/paper/1803.01271/) (2018)
- [MnasNet: Platform-Aware Neural Architecture Search for Mobile](https://mlsanity.com/paper/1807.11626/) (2018)
- [Once-for-All: Train One Network and Specialize it for Efficient Deployment](https://mlsanity.com/paper/1908.09791/) (2019)
- [ProxylessNAS: Direct Neural Architecture Search on Target Task and Hardware](https://mlsanity.com/paper/1812.00332/) (2018)
- [MCUNet: Tiny Deep Learning on IoT Devices](https://mlsanity.com/paper/2007.10319/) (2020)
- [A Critical Review of Recurrent Neural Networks for Sequence Learning](https://mlsanity.com/paper/1506.00019/) (2015)
- [FBNetV2: Differentiable Neural Architecture Search for Spatial and Channel Dimensions](https://mlsanity.com/paper/2004.05565/) (2020)
- [Binarized Neural Networks: Training Deep Neural Networks with Weights and Activations Constrained to +1 or -1](https://mlsanity.com/paper/1602.02830/) (2016)
- [MorphNet: Fast & Simple Resource-Constrained Structure Learning of Deep Networks](https://mlsanity.com/paper/1711.06798/) (2017)
- [Deep Neural Networks for Multiple Speaker Detection and Localization](https://mlsanity.com/paper/1711.11565/) (2017)

## Built on by (most cited first, 4 of 4)

- [Lightweight Neural Architecture Search for Temporal Convolutional Networks at the Edge](https://mlsanity.com/paper/2301.10281/) (2023)
- [Human Activity Recognition on Microcontrollers with Quantized and Adaptive Deep Neural Networks](https://mlsanity.com/paper/2209.00839/) (2022)
- [Multi-Complexity-Loss DNAS for Energy-Efficient and Memory-Constrained Deep Neural Networks](https://mlsanity.com/paper/2206.00302/) (2022)
- [Privacy-preserving Social Distance Monitoring on Microcontrollers with Low-Resolution Infrared Sensors and CNNs](https://mlsanity.com/paper/2204.10541/) (2022)

## Abstract (arXiv)

Temporal Convolutional Networks (TCNs) are promising Deep Learning models for time-series processing tasks. One key feature of TCNs is time-dilated convolution, whose optimization requires extensive experimentation. We propose an automatic dilation optimizer, which tackles the problem as a weight pruning on the time-axis, and learns dilation factors together with weights, in a single training. Our method reduces the model size and inference latency on a real SoC hardware target by up to 7.4x and 3x, respectively with no accuracy drop compared to a network without dilation. It also yields a rich set of Pareto-optimal TCNs starting from a single model, outperforming hand-designed solutions in both size and accuracy.