# DYNAMITE: Dynamic Interplay of Mini-Batch Size and Aggregation Frequency for Federated Learning with Static and Streaming Dataset

arXiv: https://arxiv.org/abs/2310.14906 · HTML: https://mlsanity.com/paper/2310.14906/

## Key facts

- Cited by 14 papers in the map (data through 14 Oct 2026).
- Builds on 31 of its 68 references that are in the map.
- First posted on arXiv 20 Oct 2023.

## Builds on (most cited first, 30 of 31)

- [Communication-Efficient Learning of Deep Networks from Decentralized Data](https://mlsanity.com/paper/1602.05629/) (2016)
- [Advances and Open Problems in Federated Learning](https://mlsanity.com/paper/1912.04977/) (2019)
- [Federated Optimization in Heterogeneous Networks](https://mlsanity.com/paper/1812.06127/) (2018)
- [On the Convergence of FedAvg on Non-IID Data](https://mlsanity.com/paper/1907.02189/) (2019)
- [Optimization Methods for Large-Scale Machine Learning](https://mlsanity.com/paper/1606.04838/) (2016)
- [Tackling the Objective Inconsistency Problem in Heterogeneous Federated Optimization](https://mlsanity.com/paper/2007.07481/) (2020)
- [Introduction to Online Convex Optimization](https://mlsanity.com/paper/1909.05207/) (2019)
- [Green AI](https://mlsanity.com/paper/1907.10597/) (2019)
- [Training Region-based Object Detectors with Online Hard Example Mining](https://mlsanity.com/paper/1604.03540/) (2016)
- [Linear Convergence of Gradient and Proximal-Gradient Methods Under the Polyak-Lojasiewicz Condition](https://mlsanity.com/paper/1608.04636/) (2016)
- [Adaptive Federated Learning in Resource Constrained Edge Computing Systems](https://mlsanity.com/paper/1804.05271/) (2018)
- [SAGA: A Fast Incremental Gradient Method With Support for Non-Strongly Convex Composite Objectives](https://mlsanity.com/paper/1407.0202/) (2014)
- [FedPAQ: A Communication-Efficient Federated Learning Method with Periodic Averaging and Quantization](https://mlsanity.com/paper/1909.13014/) (2019)
- [Asynchronous Federated Optimization](https://mlsanity.com/paper/1903.03934/) (2019)
- [Broadband Analog Aggregation for Low-Latency Federated Edge Learning (Extended Version)](https://mlsanity.com/paper/1812.11494/) (2018)
- [FedVision: An Online Visual Object Detection Platform Powered by Federated Learning](https://mlsanity.com/paper/2001.06202/) (2020)
- [Online Batch Selection for Faster Training of Neural Networks](https://mlsanity.com/paper/1511.06343/) (2015)
- [On Large-Cohort Training for Federated Learning](https://mlsanity.com/paper/2106.07820/) (2021)
- [Tackling System and Statistical Heterogeneity for Federated Learning with Adaptive Client Sampling](https://mlsanity.com/paper/2112.11256/) (2021)
- [Towards Flexible Device Participation in Federated Learning](https://mlsanity.com/paper/2006.06954/) (2020)
- [Cost-Effective Federated Learning Design](https://mlsanity.com/paper/2012.08336/) (2020)
- [Multi-Armed Bandit Based Client Scheduling for Federated Learning](https://mlsanity.com/paper/2007.02315/) (2020)
- [Device Scheduling with Fast Convergence for Wireless Federated Learning](https://mlsanity.com/paper/1911.00856/) (2019)
- [FedBalancer: Data and Pace Control for Efficient Federated Learning on Heterogeneous Clients](https://mlsanity.com/paper/2201.01601/) (2022)
- [Energy-Efficient Resource Management for Federated Edge Learning with CPU-GPU Heterogeneous Computing](https://mlsanity.com/paper/2007.07122/) (2020)
- [Accelerating DNN Training in Wireless Federated Edge Learning Systems](https://mlsanity.com/paper/1905.09712/) (2019)
- [Energy-Efficient Federated Edge Learning with Joint Communication and Computation Design](https://mlsanity.com/paper/2003.00199/) (2020)
- [Hierarchical Federated Learning through LAN-WAN Orchestration](https://mlsanity.com/paper/2010.11612/) (2020)
- [Taming Resource Heterogeneity In Distributed ML Training With Dynamic Batching](https://mlsanity.com/paper/2305.12213/) (2023)
- [Deep Gaussian Covariance Network](https://mlsanity.com/paper/1710.06202/) (2017)

## Built on by (most cited first, 14 of 14)

- [HASFL: Heterogeneity-aware Split Federated Learning over Edge Computing Systems](https://mlsanity.com/paper/2506.08426/) (2025)
- [Federated Continual Learning: Concepts, Challenges, and Solutions](https://mlsanity.com/paper/2502.07059/) (2025)
- [Energy-Efficient Federated Edge Learning with Streaming Data: A Lyapunov Optimization Approach](https://mlsanity.com/paper/2405.12046/) (2024)
- [SignSGD with Federated Voting](https://mlsanity.com/paper/2403.16372/) (2024)
- [Using Diffusion Models as Generative Replay in Continual Federated Learning -- What will Happen?](https://mlsanity.com/paper/2411.06618/) (2024)
- [An Information-Theoretic Analysis for Federated Learning under Concept Drift](https://mlsanity.com/paper/2506.21036/) (2025)
- [Optimal Batch-Size Control for Low-Latency Federated Learning with Device Heterogeneity](https://mlsanity.com/paper/2507.15601/) (2025)
- [Optimal Batch Allocation for Wireless Federated Learning](https://mlsanity.com/paper/2404.02395/) (2024)
- [Federated continual learning: A comprehensive survey on lifelong and privacy-preserving learning over distributed and non-stationary data](https://mlsanity.com/paper/2606.11272/) (2026)
- [TACO: Tackling Over-correction in Federated Learning with Tailored Adaptive Correction](https://mlsanity.com/paper/2504.17528/) (2025)
- [FlexFed: Mitigating Catastrophic Forgetting in Heterogeneous Federated Learning in Pervasive Computing Environments](https://mlsanity.com/paper/2505.13576/) (2025)
- [Collaborative Batch Size Optimization for Federated Learning](https://mlsanity.com/paper/2506.20511/) (2025)
- [Decentralized Rank Scheduling for Energy-Constrained Multi-Task Federated Fine-Tuning in Edge-Assisted IoV Networks](https://mlsanity.com/paper/2508.09532/) (2025)
- [Accelerating Wireless Distributed Learning via Hybrid Split and Federated Learning Optimization](https://mlsanity.com/paper/2511.19851/) (2025)

## Abstract (arXiv)

Federated Learning (FL) is a distributed learning paradigm that can coordinate heterogeneous edge devices to perform model training without sharing private data. While prior works have focused on analyzing FL convergence with respect to hyperparameters like batch size and aggregation frequency, the joint effects of adjusting these parameters on model performance, training time, and resource consumption have been overlooked, especially when facing dynamic data streams and network characteristics. This paper introduces novel analytical models and optimization algorithms that leverage the interplay between batch size and aggregation frequency to navigate the trade-offs among convergence, cost, and completion time for dynamic FL training. We establish a new convergence bound for training error considering heterogeneous datasets across devices and derive closed-form solutions for co-optimized batch size and aggregation frequency that are consistent across all devices. Additionally, we design an efficient algorithm for assigning different batch configurations across devices, improving model accuracy and addressing the heterogeneity of both data and system characteristics. Further, we propose an adaptive control algorithm that dynamically estimates network states, efficiently samples appropriate data batches, and effectively adjusts batch sizes and aggregation frequency on the fly. Extensive experiments demonstrate the superiority of our offline optimal solutions and online adaptive algorithm.