# LineupRL: Verifiable Reinforcement Learning for Time Series Captioning via Caption-to-Series Identification

arXiv: https://arxiv.org/abs/2610.01800 · HTML: https://mlsanity.com/paper/2610.01800/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Placed in RL training of language and agent models, one of 15 directions in Deep reinforcement learning methods.
- Builds on 45 of its 58 references that are in the map.
- First posted on arXiv 1 Oct 2026.

## Placed in

- Deep reinforcement learning methods / [RL training of language and agent models](https://mlsanity.com/deeprl/rl-training-of-language-and-agent-models/)

## Builds on (most cited first, 30 of 45)

- [Training language models to follow instructions with human feedback](https://mlsanity.com/paper/2203.02155/) (2022)
- [DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Models](https://mlsanity.com/paper/2402.03300/) (2024)
- [DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learning](https://mlsanity.com/paper/2501.12948/) (2025)
- [Search-R1: Training LLMs to Reason and Leverage Search Engines with Reinforcement Learning](https://mlsanity.com/paper/2503.09516/) (2025)
- [Tulu 3: Pushing Frontiers in Open Language Model Post-Training](https://mlsanity.com/paper/2411.15124/) (2024)
- [SFT Memorizes, RL Generalizes: A Comparative Study of Foundation Model Post-training](https://mlsanity.com/paper/2501.17161/) (2025)
- [Self-critical Sequence Training for Image Captioning](https://mlsanity.com/paper/1612.00563/) (2016)
- [Rubrics as Rewards: Reinforcement Learning Beyond Verifiable Domains](https://mlsanity.com/paper/2507.17746/) (2025)
- [DeepResearcher: Scaling Deep Research via Reinforcement Learning in Real-world Environments](https://mlsanity.com/paper/2504.03160/) (2025)
- [RL's Razor: Why Online Reinforcement Learning Forgets Less](https://mlsanity.com/paper/2509.04259/) (2025)
- [EmbeddingGemma: Powerful and Lightweight Text Representations](https://mlsanity.com/paper/2509.20354/) (2025)
- [Improved Image Captioning via Policy Gradient optimization of SPIDEr](https://mlsanity.com/paper/1612.00370/) (2016)
- [Reinforcement Learning with Rubric Anchors](https://mlsanity.com/paper/2508.12790/) (2025)
- [ChatTS: Aligning Time Series with LLMs via Synthetic Data for Enhanced Understanding and Reasoning](https://mlsanity.com/paper/2412.03104/) (2024)
- [Language Models Still Struggle to Zero-shot Reason about Time Series](https://mlsanity.com/paper/2404.11757/) (2024)
- [Fine-grained Image Captioning with CLIP Reward](https://mlsanity.com/paper/2205.13115/) (2022)
- [Discriminability objective for training descriptive captions](https://mlsanity.com/paper/1803.04376/) (2018)
- [Automatic Construction and Natural-Language Description of Nonparametric Regression Models](https://mlsanity.com/paper/1402.4304/) (2014)
- [Evaluating Large Language Models on Time Series Feature Understanding: A Comprehensive Taxonomy and Benchmark](https://mlsanity.com/paper/2404.16563/) (2024)
- [CapRL: Stimulating Dense Image Caption Capabilities via Reinforcement Learning](https://mlsanity.com/paper/2509.22647/) (2025)
- [Decoding Time Series with LLMs: A Multi-Agent Framework for Cross-Domain Annotation](https://mlsanity.com/paper/2410.17462/) (2024)
- [T2S: High-resolution Time Series Generation with Text-to-Series Diffusion Models](https://mlsanity.com/paper/2505.02417/) (2025)
- [VLRM: Vision-Language Models act as Reward Models for Image Captioning](https://mlsanity.com/paper/2404.01911/) (2024)
- [TimeMaster: Training Time-Series Multimodal LLMs to Reason via Reinforcement Learning](https://mlsanity.com/paper/2506.13705/) (2025)
- [Cross-Domain Image Captioning with Discriminative Finetuning](https://mlsanity.com/paper/2304.01662/) (2023)
- [BEDTime: A Unified Benchmark for Automatically Describing Time Series](https://mlsanity.com/paper/2509.05215/) (2025)
- [Truth-Conditional Captioning of Time Series Data](https://mlsanity.com/paper/2110.01839/) (2021)
- [Time Series Language Model for Descriptive Caption Generation](https://mlsanity.com/paper/2501.01832/) (2025)
- [CaTS-Bench: Can Language Models Describe Time Series?](https://mlsanity.com/paper/2509.20823/) (2025)
- [RubiCap: Rubric-Guided Reinforcement Learning for Dense Image Captioning](https://mlsanity.com/paper/2603.09160/) (2026)

## Abstract (arXiv)

Time series captioning is a fundamental step in time series understanding and can also serve as the bridge between signal and natural language. Supervised fine-tuning (SFT) relies on a larger model's captions and cannot exceed their quality. Reinforcement learning (RL) can, but its rewards were designed for other modalities and other tasks, and they transfer poorly to open-ended generation in the time series domain. We address this by proposing LineupRL, a reinforcement learning with verifiable rewards (RLVR) pipeline whose reward is caption-to-series identification. The reward model is a frozen large language model (LLM) verifier that reads the generated caption and the candidate time series as raw values, never the chart, and must pick the described time series from multiple distractors. Matching is a far lighter demand on the verifier than writing questions or judging a caption, so an off-the-shelf LLM can supply the reward. Across two captioning benchmarks, and on forecasting and reconstruction where the predictor sees only the caption, LineupRL outperforms SFT and RL baselines on every metric. The 3B vision language model (VLM) trained by LineupRL also outperforms, at 1/24 of the parameters, the 72B VLM whose captions the SFT baseline is distilled from. Our case study shows that LineupRL resists reward hacking, and that the captioner it trains both traces the trend and names the values at key points.