PsiLogic: Chaos-Aware Active Cancellation for Adam with a Fair Cross-Domain Benchmark
Not on the map. Why?
Abstract (arXiv)
Adaptive optimizers such as Adam and AdamW apply the same update rule regardless of whether training is in a chaotic early phase or near convergence. We introduce PsiLogic, an optimizer that augments Adam with a dynamic Active Cancellation Term gated by a dual exponential moving average (EMA) of scale-normalized gradient norms. The resulting chaos detector strengthens damping when gradient statistics are unstable and fades to zero as training stabilizes, providing an implicit warmup without a hand-tuned schedule. We evaluate PsiLogic against Adam, AdamW, and Lion using FairBench -- a reproducible benchmark protocol with per-optimizer learning-rate sweeps, identical initialization per seed, and Welch t-tests. On an NVIDIA H100 80GB reference run (4 arenas, 3 seeds, 2000 steps, bf16 AMP), PsiLogic achieves the best validation metric in three of four arenas: NLP perplexity 7.79 +/- 0.18 vs. 8.17 +/- 0.08 (AdamW, p = 0.049), ViT top-1 accuracy 0.244 +/- 0.006 vs. 0.223 +/- 0.002 (AdamW, p = 0.015), and ResNet top-1 accuracy 0.222 +/- 0.001 vs. 0.172 +/- 0.004 (Adam, p = 0.001). On diffusion, validation MSE is statistically tied with Adam/AdamW (p = 0.49). ResNet accuracy vs. AdamW is a numerical tie without significance at three seeds (p = 0.44). Peak GPU memory is comparable across optimizers; PsiLogic incurs 1.2--1.8x wall-clock overhead on transformer-heavy arenas (implementation-bound). We release an open-source PyTorch implementation, the full FairBench harness, and all raw CSV outputs to support independent verification.
Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 8 of its 13 references that are in the map. First posted on arXiv 5 Jul 2026.
Builds on
8 of 13 references.
References · 8
- 2018Adafactor: Adaptive Learning Rates with Sublinear Memory Costcited by 905
- 2023Symbolic Discovery of Optimization Algorithmscited by 434
- 2018Shampoo: Preconditioned Stochastic Tensor Optimizationcited by 426
- 2021High-Performance Large-Scale Image Recognition Without Normalizationcited by 360
- 2023Sophia: A Scalable Stochastic Second-order Optimizer for Language Model Pre-trainingcited by 265
- 2023Prodigy: An Expeditiously Adaptive Parameter-Free Learnercited by 133
- 2017Online Learning Rate Adaptation with Hypergradient Descentcited by 118
- 2023Learning-Rate-Free Learning by D-Adaptationcited by 113
Built on by
Not cited yet.