mlsanityData to 14 Oct 2026

Actionable Interpretability via Causal Hypergraphs: Unravelling Batch Size Effects in Deep Learning

arXiv 2506.17826 ↗First posted 21 Jun 2025cs.LG cs.AIcited by 4

Not on the map. Why?

Abstract (arXiv)

While the impact of batch size on generalisation is well studied in vision tasks, its causal mechanisms remain underexplored in graph and text domains. We introduce a hypergraph-based causal framework, HGCNet, that leverages deep structural causal models (DSCMs) to uncover how batch size influences generalisation via gradient noise, minima sharpness, and model complexity. Unlike prior approaches based on static pairwise dependencies, HGCNet employs hypergraphs to capture higher-order interactions across training dynamics. Using do-calculus, we quantify direct and mediated effects of batch size interventions, providing interpretable, causally grounded insights into optimisation. Experiments on citation networks, biomedical text, and e-commerce reviews show that HGCNet outperforms strong baselines including GCN, GAT, PI-GNN, BERT, and RoBERTa. Our analysis reveals that smaller batch sizes causally enhance generalisation through increased stochasticity and flatter minima, offering actionable interpretability to guide training strategies in deep learning. This work positions interpretability as a driver of principled architectural and optimisation choices beyond post hoc analysis.

Key facts. Cited by 4 papers in the map (data through 14 Oct 2026). Builds on 16 of its 24 references that are in the map. First posted on arXiv 21 Jun 2025.

Builds on

16 of 24 references.

References · 16

  1. 2019RoBERTa: A Robustly Optimized BERT Pretraining Approachcited by 13,372
  2. 2016Semi-Supervised Classification with Graph Convolutional Networkscited by 11,361
  3. 2017Graph Attention Networkscited by 8,923
  4. 2018How Powerful are Graph Neural Networks?cited by 4,393
  5. 2019DistilBERT, a distilled version of BERT: smaller, faster, cheaper and lightercited by 4,307
  6. 2016On Large-Batch Training for Deep Learning: Generalization Gap and Sharp Minimacited by 1,352
  7. 2015Image-based Recommendations on Styles and Substitutescited by 706
  8. 2017Sharp Minima Can Generalize For Deep Netscited by 466
  9. 2018Deep Learning on Graphs: A Surveycited by 393
  10. 2017Don't Decay the Learning Rate, Increase the Batch Sizecited by 320
  11. 2017Train longer, generalize better: closing the generalization gap in large batch training of neural networkscited by 279
  12. 2017The Marginal Value of Adaptive Gradient Methods in Machine Learningcited by 250
4 more
  1. 2018A disciplined approach to neural network hyper-parameters: Part 1 -- learning rate, batch size, momentum, and weight decaycited by 226
  2. 2021Generative Causal Explanations for Graph Neural Networkscited by 124
  3. 2020On the Almost Sure Convergence of Stochastic Gradient Descent in Non-Convex Problemscited by 80
  4. 2022Physics-Informed Graph Neural Network for Spatial-temporal Production Forecastingcited by 8

Built on by

Cited by year

Citing papers per year: 2025: 4.

4 papers.