mlsanityData to 14 Oct 2026

Regression Language Models for Code

arXiv 2509.26476 ↗First posted 30 Sep 2025cs.CL cs.AI cs.LGcited by 11

Not on the map. Why?

Abstract (arXiv)

We study code-to-metric regression: predicting numeric outcomes of code executions, a challenging task due to the open-ended nature of programming languages. While prior methods have resorted to heavy and domain-specific feature engineering, we show that a single unified Regression Language Model (RLM) using a frozen LLM encoder can simultaneously predict directly from text, (i) the memory footprint of code across multiple high-level languages such as Python and C++, (ii) the latency of Triton GPU kernels, and (iii) the accuracy and speed of trained neural networks represented in ONNX. In particular, a relatively small 300M parameter RLM based on T5Gemma, obtains >0.9 Spearman-rank on competitive programming submissions from APPS, and a single unified model achieves >0.5 average Spearman-rank across 24 different programming languages from CodeNet. Furthermore, the RLM can obtain the highest average Kendall-Tau of 0.46 on five classic NAS design spaces previously dominated by graph neural networks, and simultaneously predict architecture latencies on numerous hardware platforms.

Key facts. Cited by 11 papers in the map (data through 14 Oct 2026). Builds on 40 of its 61 references that are in the map. First posted on arXiv 30 Sep 2025.

Builds on

40 of 61 references.

References · 40

  1. 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,599
  2. 2018SentencePiece: A simple and language independent subword tokenizer and detokenizer for Neural Text Processingcited by 1,989
  3. 2022Competition-Level Code Generation with AlphaCodecited by 1,462
  4. 2021Measuring Coding Challenge Competence With APPScited by 902
  5. 2019Once-for-All: Train One Network and Specialize it for Efficient Deploymentcited by 787
  6. 2018FBNet: Hardware-Aware Efficient ConvNet Design via Differentiable Neural Architecture Searchcited by 485
  7. 2020NAS-Bench-201: Extending the Scope of Reproducible Neural Architecture Searchcited by 416
  8. 2019NAS-Bench-101: Towards Reproducible Neural Architecture Searchcited by 344
  9. 2020Neural Architecture Search without Trainingcited by 244
  10. 2021CodeNet: A Large-Scale AI for Code Dataset for Learning a Diversity of Coding Taskscited by 200
  11. 2021Zero-Cost Proxies for Lightweight NAScited by 199
  12. 2018Neural Architecture Search with Bayesian Optimisation and Optimal Transportcited by 195
28 more
  1. 2019BANANAS: Bayesian Optimization with Neural Architectures for Neural Architecture Searchcited by 179
  2. 2020AutoML-Zero: Evolving Machine Learning Algorithms From Scratchcited by 143
  3. 2023Neural Architecture Search: Insights from 1000 Paperscited by 139
  4. 2019Neural Predictor for Neural Architecture Searchcited by 118
  5. 2021How Powerful are Performance Predictors in Neural Architecture Search?cited by 90
  6. 2019On Network Design Spaces for Visual Recognitioncited by 68
  7. 2024EffiBench: Benchmarking the Efficiency of Automatically Generated Codecited by 64
  8. 2020Does Unsupervised Architecture Representation Learning Help Neural Architecture Search?cited by 59
  9. 2024From Words to Numbers: Your Large Language Model Is Secretly A Capable Regressor When Given In-Context Examplescited by 57
  10. 2020A Generic Graph-based Neural Architecture Encoding Scheme for Predictor-based NAScited by 55
  11. 2018A Survey on Compiler Autotuning using Machine Learningcited by 40
  12. 2024OmniPred: Language Models as Universal Regressorscited by 34
  13. 2021CATE: Computation-aware Neural Architecture Encoding with Transformerscited by 21
  14. 2023TpuGraphs: A Performance Prediction Dataset on Large Tensor Computational Graphscited by 16
  15. 2025Performance Prediction for Large Systems via Text-to-Text Regressioncited by 14
  16. 2018Machine Learning in Compiler Optimisationcited by 13
  17. 2020Smooth Variational Graph Embeddings for Efficient Neural Architecture Searchcited by 12
  18. 2025Decoding-based Regressioncited by 12
  19. 2024Surprisingly Strong Performance Prediction with Neural Graph Featurescited by 11
  20. 2024Encodings for Prediction-based Neural Architecture Searchcited by 9
  21. 2024On Latency Predictors for Neural Architecture Searchcited by 7
  22. 2023Multi-Predict: Few Shot Predictors For Efficient Neural Architecture Searchcited by 6
  23. 2022GENNAPE: Towards Generalized Neural Architecture Performance Estimatorscited by 5
  24. 2025Transferrable Surrogates in Expressive Neural Architecture Search Spacescited by 5
  25. 2025Concorde: Fast and Accurate CPU Performance Modeling with Compositional Analytical-ML Fusioncited by 3
  26. 2020Machine Learning Enabled Scalable Performance Prediction of Scientific Codescited by 2
  27. 2024Enhancing Source Code Representations for Deep Learning with Static Analysiscited by 2
  28. 2018AIWC: OpenCL-based Architecture-Independent Workload Characterisationcited by 1

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 3; 2026: 8.

11 papers.

  1. 2026Can We Predict Before Executing Machine Learning Agents?cited by 10
  2. 2026Aletheia: What Makes RLVR For Code Verifiers Tick?cited by 5
  3. 2026RM -RF: Reward Model for Run-Free Unit Test Evaluationcited by 2
  4. 2026RELISH: LLM REgression with a Latent Iterative State Headcited by 2
  5. 2025ONNX-Net: Towards Universal Representations and Instant Performance Prediction for Neural Architecturescited by 1
  6. 2025Beyond Token-level Supervision: Unlocking the Potential of Decoding-based Regression via Reinforcement Learningcited by 1
  7. 2026Themis: Training Robust Multilingual Code Reward Models for Flexible Multi-Criteria Scoringcited by 1
  8. 2026Reinforcement Learning over Predictive Distributions for LLM Regressioncited by 1
  9. 2025LASER: Language Model Regression for Semi-Structured Workflow Resource and Runtime Estimationcited by 0
  10. 2026Shortcut to Nowhere: Demystifying Deep Spurious Regressioncited by 0
  11. 2026DUALVIEW: Leveraging Broad and Output-Specific Evidence for LLM-based Multi-Output Text Regressioncited by 0