mlsanityData to 14 Oct 2026

On the Robustness of LLMs' Internal Representation of Code Correctness

arXiv 2608.08266 ↗First posted 8 Aug 2026cs.SE cs.AI cs.LGcited by 1

Not on the map. Why?

Abstract (arXiv)

Code generated by modern language models often reads naturally. Yet, it also often fails to implement what was asked. This should be no surprise, as research shows the models' own confidence signals are poorly calibrated with actual correctness. A promising way to assess correctness looks inside the model: by contrasting the hidden states of correct and incorrect programs, recent work captured an internal signal of code correctness that is able to judge candidate solutions better than the model's token-level or stated confidence, with no test execution. However, this signal was captured under one particular way, leaving open an important question: whether it reflects a robust property of the model or an artifact of that choice. We study this question systematically, varying how the signal is extracted from the model internals. Besides this, we also ask if the signal's quality is limited by the data used to extract it, by constructing program pairs that differ only in the fault that makes them incorrect. Our results show that no single configuration is best, and that isolating the fault does not help.

Key facts. Cited by 1 paper in the map (data through 14 Oct 2026). Builds on 47 of its 58 references that are in the map. First posted on arXiv 8 Aug 2026.

Builds on

47 of 58 references.

References · 47

  1. 2017Attention Is All You Needcited by 60,361
  2. 2021Evaluating Large Language Models Trained on Codecited by 8,320
  3. 2021Program Synthesis with Large Language Modelscited by 3,637
  4. 2023SWE-bench: Can Language Models Resolve Real-World GitHub Issues?cited by 3,373
  5. 2022Language Models (Mostly) Know What They Knowcited by 1,856
  6. 2023Is Your Code Generated by ChatGPT Really Correct? Rigorous Evaluation of Large Language Models for Code Generationcited by 1,391
  7. 2023Representation Engineering: A Top-Down Approach to AI Transparencycited by 1,336
  8. 2023Inference-Time Intervention: Eliciting Truthful Answers from a Language Modelcited by 1,302
  9. 2024Qwen2.5-Coder Technical Reportcited by 1,203
  10. 2021Probing Classifiers: Promises, Shortcomings, and Advancescited by 876
  11. 2023Just Ask for Calibration: Strategies for Eliciting Calibrated Confidence Scores from Language Models Fine-Tuned with Human Feedbackcited by 812
  12. 2022Discovering Latent Knowledge in Language Models Without Supervisioncited by 767
35 more
  1. 2023The Geometry of Truth: Emergent Linear Structure in Large Language Model Representations of True/False Datasetscited by 712
  2. 2023The Internal State of an LLM Knows When It's Lyingcited by 659
  3. 2019Designing and Interpreting Probes with Control Taskscited by 640
  4. 2024BigCodeBench: Benchmarking Code Generation with Diverse Function Calls and Complex Instructionscited by 546
  5. 2020IntelliCode Compose: Code Generation Using Transformercited by 227
  6. 2022Grounded Copilot: How Programmers Interact with Code-Generating Modelscited by 218
  7. 2025Reasoning Models Know When They're Right: Probing Hidden States for Self-Verificationcited by 159
  8. 2023Large Language Models for Code: Security Hardening and Adversarial Testingcited by 155
  9. 2020Probing the Probing Paradigm: Does Probing Accuracy Entail Task Relevance?cited by 151
  10. 2022ReCode: Robustness Evaluation of Code Generation Modelscited by 103
  11. 2020Contrastive Code Representation Learningcited by 76
  12. 2024Calibration and Correctness of Language Models for Codecited by 49
  13. 2025A Unified Understanding and Evaluation of Steering Methodscited by 46
  14. 2020Semantic Robustness of Models of Source Codecited by 45
  15. 2022Probing Classifiers are Unreliable for Concept Removal and Detectioncited by 41
  16. 2023Security Weaknesses of Copilot-Generated Code in GitHub Projects: An Empirical Studycited by 38
  17. 2025No Answer Needed: Predicting LLM Answer Accuracy from Question-Only Linear Probescited by 38
  18. 2023Emergent Representations of Program Semantics in Language Models Trained on Programscited by 30
  19. 2024Towards Understanding the Characteristics of Code Generation Errors Made by Large Language Modelscited by 30
  20. 2023Large Language Models of Code Fail at Completing Code with Potential Bugscited by 29
  21. 2024On the Universal Truthfulness Hyperplane Inside LLMscited by 24
  22. 2024The Counterfeit Conundrum: Can Code Language Models Grasp the Nuances of Their Incorrect Generations?cited by 20
  23. 2025Probing the Geometry of Truth: Consistency and Generalization of Truth Directions in LLMs Across Logical Transformations and Question Answering Taskscited by 18
  24. 2025Correctness Assessment of Code Generated by Large Language Models Using Internal Representationscited by 17
  25. 2022On Distribution Shift in Learning-based Bug Detectorscited by 15
  26. 2026LLMs Encode Their Failures: Predicting Success from Pre-Generation Activationscited by 14
  27. 2025Risk Assessment Framework for Code LLMs via Leveraging Internal Statescited by 9
  28. 2024Sifting through the Chaff: On Utilizing Execution Feedback for Ranking the Generated Code Candidatescited by 6
  29. 2025Mechanistic Interpretability of Code Correctness in LLMs via Sparse Autoencoderscited by 5
  30. 2025On LLMs' Internal Representation of Code Correctnesscited by 5
  31. 2025A Mixture of Linear Corrections Generates Secure Codecited by 4
  32. 2026The Confidence Manifold: Geometric Structure of Correctness Representations in Language Modelscited by 4
  33. 2025Localized Calibrated Uncertainty in Code Language Modelscited by 3
  34. 2026CodeCircuit: Toward Inferring LLM-Generated Code Correctness via Attribution Graphscited by 3
  35. 2025Are Sparse Autoencoders Useful for Java Function Bug Detection?cited by 2

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 1.

1 paper.

  1. 2026Hints Help But Do They Teach? Evaluating Skills Transfer in Code Generationcited by 0