DICE: Data Influence Cascade in Decentralized Learning
Not on the map. Why?
Abstract (arXiv)
Decentralized learning offers a promising approach to crowdsource data consumptions and computational workloads across geographically distributed compute interconnected through peer-to-peer networks, accommodating the exponentially increasing demands. However, proper incentives are still in absence, considerably discouraging participation. Our vision is that a fair incentive mechanism relies on fair attribution of contributions to participating nodes, which faces non-trivial challenges arising from the localized connections making influence ``cascade'' in a decentralized network. To overcome this, we design the first method to estimate Data Influence CascadE (DICE) in a decentralized environment. Theoretically, the framework derives tractable approximations of influence cascade over arbitrary neighbor hops, suggesting the influence cascade is determined by an interplay of data, communication topology, and the curvature of loss landscape. DICE also lays the foundations for applications including selecting suitable collaborators and identifying malicious behaviors. Project page is available at https://raiden-zhu.github.io/blog/2025/DICE/.
Key facts. Cited by 2 papers in the map (data through 14 Oct 2026). Builds on 87 of its 131 references that are in the map. First posted on arXiv 9 Jul 2025.
Builds on
87 of 131 references.
References · 87
- 2024The Llama 3 Herd of Modelscited by 14,715
- 2016Communication-Efficient Learning of Deep Networks from Decentralized Datacited by 7,669
- 2025DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learningcited by 6,330
- 2016Identity Mappings in Deep Residual Networkscited by 1,965
- 2017Understanding Black-box Predictions via Influence Functionscited by 1,741
- 2019Data Shapley: Equitable Valuation of Data for Machine Learningcited by 629
- 2020Personalized Federated Learning with Moreau Envelopescited by 618
- 2020Estimating Training Data Influence by Tracing Gradient Descentcited by 580
- 2023Textbooks Are All You Need II: phi-1.5 technical reportcited by 565
- 2019Certified Data Removal from Machine Learning Modelscited by 535
- 2024LESS: Selecting Influential Data for Targeted Instruction Tuningcited by 532
- 2022Beyond neural scaling laws: beating power law scaling via data pruningcited by 528
38 more
- 2020An Efficient Framework for Clustered Federated Learningcited by 511
- 2019Clustered Federated Learning: Model-Agnostic Distributed Multi-Task Optimization under Privacy Constraintscited by 503
- 2017Can Decentralized Algorithms Outperform Centralized Algorithms? A Case Study for Decentralized Parallel Stochastic Gradient Descentcited by 502
- 2021Remember What You Want to Forget: Algorithms for Machine Unlearningcited by 371
- 2020A Unified Theory of Decentralized SGD with Changing Topology and Local Updatescited by 367
- 2023Studying Large Language Model Generalization with Influence Functionscited by 326
- 2020Three Approaches for Personalization with Applications to Federated Learningcited by 325
- 2019Towards Efficient Data Valuation Based on the Shapley Valuecited by 281
- 2020Influence Functions in Deep Learning Are Fragilecited by 252
- 2020Federated Learning of a Mixture of Global and Local Modelscited by 210
- 2013On the Convergence of Decentralized Gradient Descentcited by 159
- 2017Asynchronous Decentralized Parallel Stochastic Gradient Descentcited by 159
- 2020Explaining Black Box Predictions and Unveiling Data Artifacts through Influence Functionscited by 144
- 2018D2: Decentralized Training over Decentralized Datacited by 140
- 2022Decentralized Federated Learning: Fundamentals, State of the Art, Frameworks, Trends, and Challengescited by 138
- 2022Training Data Influence Analysis and Estimation: A Surveycited by 135
- 2022Dataset Pruning: Reducing Training Data by Examining Generalization Influencecited by 133
- 2021Scaling Up Influence Functionscited by 132
- 2023DiLoCo: Distributed Low-Communication Training of Language Modelscited by 121
- 2020FastIF: Scalable Influence Functions for Efficient Model Interpretation and Debuggingcited by 120
- 2021Influence-Balanced Loss for Imbalanced Visual Classificationcited by 91
- 2022Decentralized Training of Foundation Models in Heterogeneous Environmentscited by 87
- 2022Petals: Collaborative Inference and Fine-tuning of Large Modelscited by 87
- 2023Decentralized Federated Learning: A Survey and Perspectivecited by 86
- 2024Data Shapley in One Training Runcited by 81
- 2019Measure Contribution of Participants in Federated Learningcited by 79
- 2019On Second-Order Group Influence Functions for Black-Box Predictionscited by 79
- 2021Input Similarity from the Neural Network Perspectivecited by 79
- 2020Subpopulation Data Poisoning Attackscited by 70
- 2020Optimal Complexity in Decentralized Trainingcited by 60
- 2023SWARM Parallelism: Training Large Models Can Be Surprisingly Communication-Efficientcited by 59
- 2021Exponential Graph is Provably Efficient for Decentralized Deep Trainingcited by 56
- 2017Exact Diffusion for Distributed Optimization and Learning --- Part I: Algorithm Developmentcited by 50
- 2021Consensus Control for Decentralized Deep Learningcited by 49
- 2023Dataset Distillation with Convexified Implicit Gradientscited by 46
- 2024Consent in Crisis: The Rapid Decline of the AI Data Commonscited by 46
- 2023Distributed Inference and Fine-tuning of Large Language Models Over The Internetcited by 42
- 2021RelaySum for Decentralized Deep Learning on Heterogeneous Datacited by 39
50 most cited of 87 All 87 (JSON)
Built on by
Citing papers per year: 2025: 2.
2 papers.