# Aggregation in the Mirror Space (AIMS): Fast, Accurate Distributed Machine Learning in Military Settings

arXiv: https://arxiv.org/abs/2210.16181 · HTML: https://mlsanity.com/paper/2210.16181/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 10 of its 22 references that are in the map.
- First posted on arXiv 28 Oct 2022.

## Builds on (most cited first, 10 of 10)

- [Convex Optimization: Algorithms and Complexity](https://mlsanity.com/paper/1405.4980/) (2014)
- [Ray: A Distributed Framework for Emerging AI Applications](https://mlsanity.com/paper/1712.05889/) (2017)
- [Can Decentralized Algorithms Outperform Centralized Algorithms? A Case Study for Decentralized Parallel Stochastic Gradient Descent](https://mlsanity.com/paper/1705.09056/) (2017)
- [A Survey on Distributed Machine Learning](https://mlsanity.com/paper/1912.09789/) (2019)
- [Quasi-Global Momentum: Accelerating Decentralized Deep Learning on Heterogeneous Data](https://mlsanity.com/paper/2102.04761/) (2021)
- [Decentralized gradient methods: does topology matter?](https://mlsanity.com/paper/2002.12688/) (2020)
- [Minimizing Uniformly Convex Functions by Cubic Regularization of Newton Method](https://mlsanity.com/paper/1905.02671/) (2019)
- [TornadoAggregate: Accurate and Scalable Federated Learning via the Ring-Based Architecture](https://mlsanity.com/paper/2012.03214/) (2020)
- [Distributed Mirror Descent for Online Composite Optimization](https://mlsanity.com/paper/2004.00837/) (2020)
- [Optimal Distributed Stochastic Mirror Descent for Strongly Convex Optimization](https://mlsanity.com/paper/1610.04702/) (2016)

## Abstract (arXiv)

Distributed machine learning (DML) can be an important capability for modern military to take advantage of data and devices distributed at multiple vantage points to adapt and learn. The existing distributed machine learning frameworks, however, cannot realize the full benefits of DML, because they are all based on the simple linear aggregation framework, but linear aggregation cannot handle the textitdivergence challenges arising in military settings: the learning data at different devices can be heterogeneous (textiti.e., Non-IID data), leading to model divergence, but the ability for devices to communicate is substantially limited (textiti.e., weak connectivity due to sparse and dynamic communications), reducing the ability for devices to reconcile model divergence. In this paper, we introduce a novel DML framework called aggregation in the mirror space (AIMS) that allows a DML system to introduce a general mirror function to map a model into a mirror space to conduct aggregation and gradient descent. Adapting the convexity of the mirror function according to the divergence force, AIMS allows automatic optimization of DML. We conduct both rigorous analysis and extensive experimental evaluations to demonstrate the benefits of AIMS. For example, we prove that AIMS achieves a loss of Oleft((fracmr+1T)frac1rright) after T network-wide updates, where m is the number of devices and r the convexity of the mirror function, with existing linear aggregation frameworks being a special case with r=2. Our experimental evaluations using EMANE (Extendable Mobile Ad-hoc Network Emulator) for military communications settings show similar results: AIMS can improve DML convergence rate by up to 57% and scale well to more devices with weak connectivity, all with little additional computation overhead compared to traditional linear aggregation.