# Trait-Aware Policy Optimization for Autoregressive Multi-Trait Essay Scoring

arXiv: https://arxiv.org/abs/2605.25731 · HTML: https://mlsanity.com/paper/2605.25731/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 26 of its 56 references that are in the map.
- First posted on arXiv 25 May 2026.

## Builds on (most cited first, 26 of 26)

- [Training language models to follow instructions with human feedback](https://mlsanity.com/paper/2203.02155/) (2022)
- [Proximal Policy Optimization Algorithms](https://mlsanity.com/paper/1707.06347/) (2017)
- [Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer](https://mlsanity.com/paper/1910.10683/) (2019)
- [DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Models](https://mlsanity.com/paper/2402.03300/) (2024)
- [Let's Verify Step by Step](https://mlsanity.com/paper/2305.20050/) (2023)
- [DAPO: An Open-Source LLM Reinforcement Learning System at Scale](https://mlsanity.com/paper/2503.14476/) (2025)
- [Fine-Tuning Language Models from Human Preferences](https://mlsanity.com/paper/1909.08593/) (2019)
- [Understanding R1-Zero-Like Training: A Critical Perspective](https://mlsanity.com/paper/2503.20783/) (2025)
- [Solving math word problems with process- and outcome-based feedback](https://mlsanity.com/paper/2211.14275/) (2022)
- [CPPO: Accelerating the Training of Group Relative Policy Optimization-Based Reasoning Models](https://mlsanity.com/paper/2503.22342/) (2025)
- [Automatic Text Scoring Using Neural Networks](https://mlsanity.com/paper/1606.04289/) (2016)
- [On the Use of BERT for Automated Essay Scoring: Joint Learning of Multi-Scale Essay Representation](https://mlsanity.com/paper/2205.03835/) (2022)
- [Unleashing Large Language Models' Proficiency in Zero-shot Essay Scoring](https://mlsanity.com/paper/2404.04941/) (2024)
- [Automated essay scoring with string kernels and word embeddings](https://mlsanity.com/paper/1804.07954/) (2018)
- [CAPO: Towards Enhancing LLM Reasoning through Generative Credit Assignment](https://mlsanity.com/paper/2508.02298/) (2025)
- [Many Hands Make Light Work: Using Essay Traits to Automatically Score Essays](https://mlsanity.com/paper/2102.00781/) (2021)
- [GRPO-λ: Credit Assignment improves LLM Reasoning](https://mlsanity.com/paper/2510.00194/) (2025)
- [Rationale Behind Essay Scores: Enhancing S-LLM's Multi-Trait Essay Scoring with Rationale Generated by LLMs](https://mlsanity.com/paper/2410.14202/) (2024)
- [Outcome-Grounded Advantage Reshaping for Fine-Grained Credit Assignment in Mathematical Reasoning](https://mlsanity.com/paper/2601.07408/) (2026)
- [Autoregressive Multi-trait Essay Scoring via Reinforcement Learning with Scoring-aware Multiple Rewards](https://mlsanity.com/paper/2409.17472/) (2024)
- [TRATES: Trait-Specific Rubric-Assisted Cross-Prompt Essay Scoring](https://mlsanity.com/paper/2505.14577/) (2025)
- [Proximal Policy Optimization and its Dynamic Version for Sequence Generation](https://mlsanity.com/paper/1808.07982/) (2018)
- [DGPO: Distribution Guided Policy Optimization for Fine Grained Credit Assignment](https://mlsanity.com/paper/2605.03327/) (2026)
- [Teach-to-Reason with Scoring: Self-Explainable Rationale-Driven Multi-Trait Essay Scoring](https://mlsanity.com/paper/2502.20748/) (2025)
- [Transformer-based Joint Modelling for Automatic Essay Scoring and Off-Topic Detection](https://mlsanity.com/paper/2404.08655/) (2024)
- [Enhancing Automated Essay Scoring With Three Techniques: Two-Stage Fine-Tuning, Score Alignment, and Self-Training](https://mlsanity.com/paper/2602.01747/) (2026)

## Built on by (most cited first, 1 of 1)

- [A Unified Framework to Elicit Structured Feedback for Interpretable Multi-Trait Essay Scoring](https://mlsanity.com/paper/2608.28407/) (2026)

## Abstract (arXiv)

Multi-trait essay scoring aims to provide fine-grained evaluation of writing quality across multiple dimensions. However, how to effectively post-train autoregressive scoring models remains underexplored. In this paper, we propose Trait-Aware Policy Optimization (TAPO), a post-training framework tailored to autoregressive multi-trait scoring. Our method decomposes rewards along both the sample and trait dimensions, combining global scoring consistency, trait-level accuracy, format validity, and inter-trait dependency preservation. In addition, we use enhanced prompts throughout training by incorporating original prompt texts and trait descriptions, providing richer semantic information for trait-specific score generation. Experiments across multiple backbone models show that our method consistently improves multi-trait scoring performance over supervised fine-tuning and scalar-reward optimization baselines, demonstrating the effectiveness and transferability of trait-aware post-training for essay scoring.