# Enabling Fast and Flexible Distributed Deep Learning with Programmable Switches

arXiv: https://arxiv.org/abs/2205.05243 · HTML: https://mlsanity.com/paper/2205.05243/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 11 May 2022.

## Built on by (most cited first, 1 of 1)

- [Expediting In-Network Federated Learning by Voting-Based Consensus Model Compression](https://mlsanity.com/paper/2402.03815/) (2024)

## Abstract (arXiv)

Deep learning has been used in a wide range of areas and made a huge breakthrough. With the ever-increasing model size and train-ing data volume, distributed deep learning emerges which utilizes a cluster to train a model in parallel. Unfortunately, the performance is often far from linear speedup due to the communication overhead between cluster nodes. To address this challenge, this paper designs and implements Libra, a network aggregator, that utilizes in-network computation to optimize the communication for distributed DL training in two aspects: 1) reduce active connections and 2) aggregate exchanged network packets. We implemented our Libra on Intel Tofino switches, customized a lightweight host stack and integrated it into an open-source training framework PS-lite. The experimental result shows that our Libra can achieve 1.5~4 times speedup.