Model-Centric Diagnostics: A Framework for Internal State Readouts
Not on the map. Why?
Abstract (arXiv)
We present a model-centric diagnostic framework that treats training state as a latent variable and unifies a family of internal readouts -- head-gradient norms, confidence, entropy, margin, and related signals -- as anchor-relative projections of that state. A preliminary version of this work introduced a head-gradient probe for checkpoint selection. In this version, we focus on the unifying perspective and structural diagnostics; full algorithmic details, theoretical analysis, and experimental validation will appear in a forthcoming paper. We outline the conceptual scaffold: any prediction head induces a local loss landscape whose geometry (gradient magnitude, curvature, sharpness) reflects how well the upstream features are aligned with the task. Different readout choices -- gradient norms, softmax entropy, predictive margin -- correspond to different projections of this geometry, each with complementary strengths. The framework suggests that checkpoint selection, early stopping, and lightweight architecture pre-screening can all be viewed as querying the same underlying state through different lenses. Illustrative experiments on ImageNet classification and COCO detection/segmentation hint at the practical potential; rigorous benchmarks and ablations are deferred to the full paper.
Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 18 of its 30 references that are in the map. First posted on arXiv 23 Jan 2026.
Builds on
18 of 30 references.
References · 18
- 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 30,186
- 2020Denoising Diffusion Probabilistic Modelscited by 21,076
- 2021Swin Transformer: Hierarchical Vision Transformer using Shifted Windowscited by 12,139
- 2015Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networkscited by 9,639
- 2019EfficientNet: Rethinking Model Scaling for Convolutional Neural Networkscited by 6,442
- 2017On Calibration of Modern Neural Networkscited by 4,960
- 2018MobileNetV2: Inverted Residuals and Linear Bottleneckscited by 4,795
- 2020Training data-efficient image transformers & distillation through attentioncited by 4,586
- 2021Masked-attention Mask Transformer for Universal Image Segmentationcited by 2,351
- 2020Sharpness-Aware Minimization for Efficiently Improving Generalizationcited by 1,417
- 2016On Large-Batch Training for Deep Learning: Generalization Gap and Sharp Minimacited by 1,352
- 2019Fantastic Generalization Measures and Where to Find Themcited by 513
6 more
- 2020Pruning neural networks without any data by iteratively conserving synaptic flowcited by 479
- 2021Zero-Cost Proxies for Lightweight NAScited by 199
- 2021You Only Look at One Sequence: Rethinking Transformer in Vision through Object Detectioncited by 170
- 2021Neural Architecture Search on ImageNet in Four GPU Hours: A Theoretically Inspired Perspectivecited by 169
- 2024What is YOLOv8: An In-Depth Exploration of the Internal Features of the Next-Generation Object Detectorcited by 45
- 2017Early Stopping without a Validation Setcited by 39
Built on by
Not cited yet.