Meta-TTRL: A Metacognitive Framework for Self-Improving Test-Time Reinforcement Learning in Unified Multimodal Models
On the map: LLM reasoning → Multimodal reasoning
Abstract (arXiv)
Existing test-time scaling (TTS) methods for unified multimodal models (UMMs) in text-to-image (T2I) generation primarily rely on search or sampling strategies that produce only instance-level improvements, limiting the ability to learn from prior inferences and accumulate knowledge across similar prompts. To overcome these limitations, we propose Meta-TTRL, a metacognitive test-time reinforcement learning framework. Meta-TTRL performs test-time parameter optimization guided by model-intrinsic monitoring signals derived from the meta-knowledge of UMMs, achieving self-improvement and capability-level improvement at test time. Extensive experiments demonstrate that Meta-TTRL generalizes well across three representative UMMs, including Janus-Pro-7B, BAGEL, and Qwen-Image, achieving significant gains on compositional reasoning tasks and multiple T2I benchmarks with limited data. We provide the first comprehensive analysis to investigate the potential of test-time reinforcement learning (TTRL) for T2I generation in UMMs. Our analysis further reveals a key insight underlying effective TTRL: metacognitive synergy, where monitoring signals align with the model's optimization regime to enable self-improvement.
Key facts. Cited by 4 papers in the map (data through 7 Oct 2026). Placed in Multimodal reasoning, one of 17 directions in LLM reasoning. Builds on 26 of its 28 references that are in the map. First posted on arXiv 16 Mar 2026.
Builds on
26 of 28 references.
Closest ancestors · 3
Same direction, newest first
- 2026UniT: Unified Multimodal Chain-of-Thought Test-time Scaling12 Feb
- 2025T2I-R1: Reinforcing Image Generation with Collaborative Semantic-level and Token-level CoT1 May
- 2025Can We Generate Images with CoT? Let's Verify and Reinforce Image Generation Step by Step23 Jan
Other references · 23
- 2020Language Models are Few-Shot Learnerscited by 32,199
- 2023GPT-4 Technical Reportcited by 18,132
- 2025Qwen3-VL Technical Reportcited by 2,534
- 2016Making the V in VQA Matter: Elevating the Role of Image Understanding in Visual Question Answeringcited by 2,498
- 2023GenEval: An Object-Focused Framework for Evaluating Text-to-Image Alignmentcited by 946
- 2024Chameleon: Mixed-Modal Early-Fusion Foundation Modelscited by 904
- 2025Emerging Properties in Unified Multimodal Pretrainingcited by 855
- 2024Emu3: Next-Token Prediction is All You Needcited by 761
- 2024Show-o: One Single Transformer to Unify Multimodal Understanding and Generationcited by 734
- 2022GIT: A Generative Image-to-text Transformer for Vision and Languagecited by 536
- 2024ELLA: Equip Diffusion Models with LLM for Enhanced Semantic Alignmentcited by 455
- 2023DreamLLM: Synergistic Multimodal Comprehension and Creationcited by 296
11 more
- 2023T2I-CompBench++: An Enhanced and Comprehensive Benchmark for Compositional Text-to-image Generationcited by 253
- 2025TTRL: Test-Time Reinforcement Learningcited by 253
- 2025Unified Reward Model for Multimodal Understanding and Generationcited by 194
- 2025Emu3.5: Native Multimodal Models are World Learnerscited by 119
- 2025TIIF-Bench: How Does Your T2I Model Follow Your Instructions?cited by 64
- 2025UniGen: Enhanced Training & Test-Time Strategies for Unified Multimodal Understanding and Generationcited by 32
- 2023MMoE: Enhancing Multimodal Models with Mixtures of Multimodal Interaction Expertscited by 21
- 2025Enhancing Reward Models for High-quality Image Generation: Beyond Text-Image Alignmentcited by 14
- 2025ImAgent: A Unified Multimodal Agent Framework for Test-Time Scalable Image Generationcited by 14
- 2025TTRV: Test-Time Reinforcement Learning for Vision Language Modelscited by 8
- 2025StableSketcher: Enhancing Diffusion Model for Pixel-based Sketch Generation via Visual Question Answering Feedbackcited by 1
Built on by
Citing papers per year: 2026: 4.
4 papers.
- 2026Detecting and Mitigating the Correct-Answer Extinction Window in Test-Time Reinforcement Learning with Majority Votingcited by 5
- 2026PosterText: Towards Unified Visual Text Generation and Editing for E-commerce Postercited by 0
- 2026Unsupervised Post-Training of Foundation Models: A Surveycited by 0
- 2026UniEvo-VL: An On-policy Self-Distillation Training Recipe for Multimodal Model Self-improvementcited by 0
- 2026UniEvo-VL: An On-policy Self-Distillation Training Recipe for Multimodal Model Self-improvement30 Sep 2026
- 2026Unsupervised Post-Training of Foundation Models: A Survey25 Aug 2026
- 2026PosterText: Towards Unified Visual Text Generation and Editing for E-commerce Poster17 Aug 2026
- 2026Detecting and Mitigating the Correct-Answer Extinction Window in Test-Time Reinforcement Learning with Majority Voting19 May 2026
Similar, not cited
Close in wording, no citation either way
- 2026TTRSD: Test-Time Reinforcement Learning with Self-Distillation for Vision-Language Models
- 2026Harnessing Image Question Dependence for Better VLM Test-time Reinforcement Learning
- 2025Extending Test-Time Scaling: A 3D Perspective with Context, Batch, and Turn
- 2024Diving into Self-Evolving Training for Multimodal Reasoning
- 2025Adapting Like Humans: A Metacognitive Agent with Test-time Reasoning
- 2026Test-Time Scaling in Multimodal Foundation Models: A Comprehensive Survey of Generation and Reasoning
- 2025VideoChat-R1.5: Visual Test-Time Scaling to Reinforce Multimodal Reasoning by Iterative Perception
- 2026Mimic Human Cognition, Master Multi-Image Reasoning: A Meta-Action Framework for Enhanced Visual Understanding