# Flare: Flexible In-Network Allreduce

arXiv: https://arxiv.org/abs/2106.15565 · HTML: https://mlsanity.com/paper/2106.15565/

## Key facts

- Cited by 9 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 29 Jun 2021.

## Built on by (most cited first, 9 of 9)

- [ASTRA-sim2.0: Modeling Hierarchical Networks and Disaggregated Systems for Large-model Training at Scale](https://mlsanity.com/paper/2303.14006/) (2023)
- [Themis: A Network Bandwidth-Aware Collective Scheduling Policy for Distributed Training of DL Models](https://mlsanity.com/paper/2110.04478/) (2021)
- [HammingMesh: A Network Topology for Large-Scale Deep Learning](https://mlsanity.com/paper/2209.01346/) (2022)
- [Swing: Short-cutting Rings for Higher Bandwidth Allreduce](https://mlsanity.com/paper/2401.09356/) (2024)
- [ForestColl: Throughput-Optimal Collective Communications on Heterogeneous Network Fabrics](https://mlsanity.com/paper/2402.06787/) (2024)
- [SparDL: Distributed Deep Learning Training with Efficient Sparse Communication](https://mlsanity.com/paper/2304.00737/) (2023)
- [In-Network Collective Operations: Game Changer or Challenge for AI Workloads?](https://mlsanity.com/paper/2601.19132/) (2026)
- [A Switch-Centric In-Network Architecture for Accelerating LLM Inference in Shared-Memory Network](https://mlsanity.com/paper/2603.28239/) (2026)
- [Accelerating MoE with Dynamic In-Switch Computing on Multi-GPUs](https://mlsanity.com/paper/2605.05607/) (2026)

## Abstract (arXiv)

The allreduce operation is one of the most commonly used communication routines in distributed applications. To improve its bandwidth and to reduce network traffic, this operation can be accelerated by offloading it to network switches, that aggregate the data received from the hosts, and send them back the aggregated result. However, existing solutions provide limited customization opportunities and might provide suboptimal performance when dealing with custom operators and data types, with sparse data, or when reproducibility of the aggregation is a concern. To deal with these problems, in this work we design a flexible programmable switch by using as a building block PsPIN, a RISC-V architecture implementing the sPIN programming model. We then design, model, and analyze different algorithms for executing the aggregation on this architecture, showing performance improvements compared to state-of-the-art approaches.