mlsanityData to 7 Oct 2026

Cross-Entropy Is Load-Bearing: A Pre-Registered Scope Test of the K-Way Energy Probe on Bidirectional Predictive Coding

arXiv 2604.21286 ↗First posted 23 Apr 2026cs.CL cs.AI cs.LGcited by 1

Not on the map. Why?

Abstract (arXiv)

Cacioli (2026) showed that the K-way energy probe on standard discriminative predictive coding networks reduces approximately to a monotone function of the log-softmax margin. The reduction rests on five assumptions, including cross-entropy (CE) at the output and effectively feedforward inference dynamics. This pre-registered study tests the reduction's sensitivity to CE removal using two conditions: standard PC trained with MSE instead of CE, and bidirectional PC (bPC; Oliviers, Tang & Bogacz, 2025). Across 10 seeds on CIFAR-10 with a matched 2.1M-parameter backbone, we find three results. The negative result replicates on standard PC: the probe sits below softmax (Delta = -0.082, p < 10^-6). On bPC the probe exceeds softmax across all 10 seeds (Delta = +0.008, p = 0.000027), though a pre-registered manipulation check shows that bPC does not produce materially greater latent movement than standard PC at this scale (ratio 1.6, threshold 10). Removing CE alone without changing inference dynamics halves the probe-softmax gap (Delta_MSE = -0.037 vs Delta_stdPC = -0.082). CE is a major empirically load-bearing component of the decomposition at this scale. CE training produces output logit norms approximately 15x larger than MSE or bPC training. A post-hoc temperature scaling ablation decomposes the probe-softmax gap into two components: approximately 66% is attributable to logit-scale effects removable by temperature rescaling, and approximately 34% reflects a scale-invariant ranking advantage of CE-trained representations. We use "metacognitive" operationally to denote Type-2 discrimination of a readout over its own Type-1 correctness, not to imply human-like introspective access.

Key facts. Cited by 1 paper in the map (data through 7 Oct 2026). Builds on 6 of its 15 references that are in the map. First posted on arXiv 23 Apr 2026.

Builds on

6 of 15 references.

References · 6

  1. 2017On Calibration of Modern Neural Networkscited by 4,860
  2. 2022Mitigating Neural Network Overconfidence with Logit Normalizationcited by 251
  3. 2020Predictive Coding Approximates Backprop along Arbitrary Computation Graphscited by 89
  4. 2024Benchmarking Predictive Coding Networks -- Made Simplecited by 29
  5. 2025μPC: Scaling Predictive Coding to 100+ Layer Networkscited by 14
  6. 2026K-Way Energy Probes for Metacognition Reduce to Softmax in Discriminative Predictive Coding Networkscited by 1

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2026: 1.

1 paper.