# Weak-to-Strong Learning in Decision Making

arXiv: https://arxiv.org/abs/2607.18467 · HTML: https://mlsanity.com/paper/2607.18467/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 16 of its 80 references that are in the map.
- First posted on arXiv 20 Jul 2026.

## Builds on (most cited first, 16 of 16)

- [Scaling Laws for Neural Language Models](https://mlsanity.com/paper/2001.08361/) (2020)
- [DistilBERT, a distilled version of BERT: smaller, faster, cheaper and lighter](https://mlsanity.com/paper/1910.01108/) (2019)
- [Scaling Laws for Transfer](https://mlsanity.com/paper/2102.01293/) (2021)
- [Great Models Think Alike and this Undermines AI Oversight](https://mlsanity.com/paper/2502.04313/) (2025)
- [Softmax is 1/2-Lipschitz: A tight bound across all ℓp norms](https://mlsanity.com/paper/2510.23012/) (2025)
- [Estimate-Then-Optimize versus Integrated-Estimation-Optimization versus Sample Average Approximation: A Stochastic Dominance Perspective](https://mlsanity.com/paper/2304.06833/) (2023)
- [Discrepancies are Virtue: Weak-to-Strong Generalization through Lens of Intrinsic Dimension](https://mlsanity.com/paper/2502.05075/) (2025)
- [Active Learning For Contextual Linear Optimization: A Margin-Based Approach](https://mlsanity.com/paper/2305.06584/) (2023)
- [Design and Scheduling of an AI-based Queueing System](https://mlsanity.com/paper/2406.06855/) (2024)
- [On the sample complexity of semi-supervised multi-objective learning](https://mlsanity.com/paper/2508.17152/) (2025)
- [Decision-Focused Sequential Experimental Design: A Directional Uncertainty-Guided Approach](https://mlsanity.com/paper/2602.05340/) (2026)
- [A Solver-Free Training Method for Predict-then-Optimize](https://mlsanity.com/paper/2606.19587/) (2026)
- [Multi-Task Dynamic Pricing in Credit Market with Contextual Information](https://mlsanity.com/paper/2410.14839/) (2024)
- [Does Weak-to-strong Generalization Happen under Spurious Correlations?](https://mlsanity.com/paper/2509.24005/) (2025)
- [The Sign Estimator: LLM Alignment in the Face of Choice Heterogeneity](https://mlsanity.com/paper/2510.23965/) (2025)
- [Coupled Training with Privileged Information and Unlabeled Data](https://mlsanity.com/paper/2605.23268/) (2026)

## Abstract (arXiv)

Many operational decisions rely on predictive models that estimate uncertain outcomes conditional on observable contexts. Training such models, however, often faces a fundamental data asymmetry: labeled outcomes are scarce or costly to obtain, while contextual covariates are abundant. Motivated by this data asymmetry, we develop a decision-aware weak-to-strong (W2S) framework that leverages both labeled and unlabeled data to improve contextual stochastic optimization. Specifically, we first train a weak model using limited labeled data and then use it to generate predicted outcome distributions on unlabeled contexts. These distributions provide soft supervision for training a strong model. We establish a non-asymptotic upper bound on the excess decision risk of W2S and a complementary lower bound for a strong-only benchmark. Their comparison yields explicit sufficient conditions under which W2S improves downstream decision performance. The key quantity is the correlation dimension between the weak and strong feature representations: when it is small, abundant unlabeled data reduce the effect of teacher errors along non-overlapping directions. A synthetic newsvendor experiment and a comment moderation experiment based on real-world data provide empirical evidence consistent with the theory.