MetaLint: Easy-to-Hard Generalization for Code Linting
Not on the map. Why?
Abstract (arXiv)
Large language models excel at code generation but struggle with code linting, particularly in generalizing to unseen or evolving best practices beyond those observed during training. We introduce MetaLint, a meta-learning framework that formulates code linting as an instruction-following task, where a model evaluates whether code adheres to a natural language specification of best practices. In contrast to prior work that trains models to detect violations from a fixed set of best practices, MetaLint evaluates code against a provided natural language specification, enabling test-time control over which practices to enforce and generalization to unseen or evolving rules without retraining. We demonstrate that models trained solely on synthetic data generated from automatic linters still generalize to harder, context-dependent best practices for which such linters are not available. To evaluate generalization beyond such easy signals, we introduce a human-curated benchmark of hard best practices inspired by Python Enhancement Proposals (PEPs). On this benchmark, MetaLint substantially improves performance without explicit fine-tuning on target best practices and exhibits strong, easy-to-hard generalization. Qwen3-4B achieves a 2.7x detection F-score gain (25.9% -> 70.4%), the highest recall, and a 26.7% localization F-score, matching larger models such as o3-mini. These gains generalize across programming languages, model families, scales, reasoning settings, and linter sources. We release the code and benchmark to support reproducibility and future work.
Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 42 of its 61 references that are in the map. First posted on arXiv 15 Jul 2025.
Builds on
42 of 61 references.
References · 42
- 2020Language Models are Few-Shot Learnerscited by 32,419
- 2017Proximal Policy Optimization Algorithmscited by 15,215
- 2023Llama 2: Open Foundation and Fine-Tuned Chat Modelscited by 13,028
- 2024DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Modelscited by 9,379
- 2023Direct Preference Optimization: Your Language Model is Secretly a Reward Modelcited by 8,065
- 2024GPT-4o System Cardcited by 4,588
- 2021Finetuned Language Models Are Zero-Shot Learnerscited by 3,780
- 2022Scaling Instruction-Finetuned Language Modelscited by 3,024
- 2021Multitask Prompted Training Enables Zero-Shot Task Generalizationcited by 1,470
- 2024Qwen2.5-Coder Technical Reportcited by 1,203
- 2022STaR: Bootstrapping Reasoning With Reasoningcited by 940
- 2022Super-NaturalInstructions: Generalization via Declarative Instructions on 1600+ NLP Taskscited by 868
30 more
- 2021Cross-Task Generalization via Natural Language Crowdsourcing Instructionscited by 699
- 2024StarCoder 2 and The Stack v2: The Next Generationcited by 502
- 2024Generative Verifiers: Reward Modeling as Next-Token Predictioncited by 480
- 2025Small Language Models are the Future of Agentic AIcited by 270
- 2022OPT-IML: Scaling Language Model Instruction Meta Learning through the Lens of Generalizationcited by 236
- 2024Easy-to-Hard Generalization: Scalable Alignment Beyond Human Supervisioncited by 96
- 2025Curriculum Reinforcement Learning from Easy to Hard Tasks Improves LLM Reasoningcited by 92
- 2025Can Large Reasoning Models Self-Train?cited by 68
- 2023Understanding the Effectiveness of Large Language Models in Detecting Security Vulnerabilitiescited by 59
- 2024RS-DPO: A Hybrid Rejection Sampling and Direct Preference Optimization Method for Alignment of Large Language Modelscited by 48
- 2024SELF-GUIDE: Better Task-Specific Instruction Following via Self-Synthetic Finetuningcited by 32
- 2016Differentiable Programs with Neural Librariescited by 30
- 2025RewardAnything: Generalizable Principle-Following Reward Modelscited by 28
- 2024BudgetMLAgent: A Cost-Effective LLM Multi-Agent system for Automating Machine Learning Taskscited by 19
- 2022How Many Data Samples is an Additional Instruction Worth?cited by 16
- 2023CrossCodeBench: Benchmarking Cross-Task Generalization of Source Code Modelscited by 16
- 2024Instruction Diversity Drives Generalization To Unseen Taskscited by 16
- 2024AI-Assisted Assessment of Coding Practices in Modern Code Reviewcited by 16
- 2022Learning Instructions with Unlabeled Data for Zero-Shot Cross-Task Generalizationcited by 15
- 2024Efficient Model-agnostic Alignment via Bayesian Persuasioncited by 8
- 2024CRScore: Grounding Automated Evaluation of Code Review Comments in Code Claims and Smellscited by 8
- 2025LintLLM: An Open-Source Verilog Linting Framework Based on Large Language Modelscited by 7
- 2025Beyond Single-Task: Robust Multi-Task Length Generalization for LLMscited by 7
- 2025Combining Large Language Models with Static Analyzers for Code Review Generationcited by 6
- 2025Minimizing False Positives in Static Bug Detection via LLM-Enhanced Path Feasibility Analysiscited by 4
- 2024Refactoring to Pythonic Idioms: A Hybrid Knowledge-Driven Approach Leveraging Large Language Modelscited by 3
- 2025CRScore++: Reinforcement Learning with Verifiable Tool and AI Feedback for Code Reviewcited by 3
- 2024Guiding Through Complexity: What Makes Good Supervision for Hard Math Reasoning Tasks?cited by 2
- 2024Code Linting using Language Modelscited by 1
- 2025Quantum Program Linting with LLMs: Emerging Results from a Comparative Studycited by 1
Built on by
Not cited yet.