# Do Time-Series QA Systems Read the Time Series? Evidence Use and Reasoning Reliability

arXiv: https://arxiv.org/abs/2610.05686 · HTML: https://mlsanity.com/paper/2610.05686/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 27 of its 30 references that are in the map.
- First posted on arXiv 5 Oct 2026.

## Builds on (most cited first, 27 of 27)

- [Language Models Don't Always Say What They Think: Unfaithful Explanations in Chain-of-Thought Prompting](https://mlsanity.com/paper/2305.04388/) (2023)
- [Emerging Properties in Unified Multimodal Pretraining](https://mlsanity.com/paper/2505.14683/) (2025)
- [Measuring Faithfulness in Chain-of-Thought Reasoning](https://mlsanity.com/paper/2307.13702/) (2023)
- [Towards Faithfully Interpretable NLP Systems: How should we define and evaluate faithfulness?](https://mlsanity.com/paper/2004.03685/) (2020)
- [ERASER: A Benchmark to Evaluate Rationalized NLP Models](https://mlsanity.com/paper/1911.03429/) (2019)
- [GIFT-Eval: A Benchmark For General Time Series Forecasting Model Evaluation](https://mlsanity.com/paper/2410.10393/) (2024)
- [Are Language Models Actually Useful for Time Series Forecasting?](https://mlsanity.com/paper/2406.16964/) (2024)
- [ChatTS: Aligning Time Series with LLMs via Synthetic Data for Enhanced Understanding and Reasoning](https://mlsanity.com/paper/2412.03104/) (2024)
- [ChatTime: A Unified Multimodal Time Series Foundation Model Bridging Numerical and Textual Data](https://mlsanity.com/paper/2412.11376/) (2024)
- [Language Models Still Struggle to Zero-shot Reason about Time Series](https://mlsanity.com/paper/2404.11757/) (2024)
- [Context is Key: A Benchmark for Forecasting with Essential Textual Information](https://mlsanity.com/paper/2410.18959/) (2024)
- [Time-MQA: Time Series Multi-Task Question Answering with Context Enhancement](https://mlsanity.com/paper/2503.01875/) (2025)
- [It's TIME: Towards the Next Generation of Time Series Forecasting Benchmarks](https://mlsanity.com/paper/2602.12147/) (2026)
- [TimeOmni-1: Incentivizing Complex Reasoning with Time Series in Large Language Models](https://mlsanity.com/paper/2509.24803/) (2025)
- [FaithCoT-Bench: Benchmarking Instance-Level Faithfulness of Chain-of-Thought Reasoning](https://mlsanity.com/paper/2510.04040/) (2025)
- [TSRBench: A Comprehensive Multi-task Multi-modal Time Series Reasoning Benchmark for Generalist Models](https://mlsanity.com/paper/2601.18744/) (2026)
- [SciTS: Scientific Time Series Understanding and Generation with LLMs](https://mlsanity.com/paper/2510.03255/) (2025)
- [When Does Multimodality Lead to Better Time Series Forecasting?](https://mlsanity.com/paper/2506.21611/) (2025)
- [TSAQA: Time Series Analysis Question And Answering Benchmark](https://mlsanity.com/paper/2601.23204/) (2026)
- [RFEval: Benchmarking Reasoning Faithfulness under Counterfactual Reasoning Intervention in Large Reasoning Models](https://mlsanity.com/paper/2602.17053/) (2026)
- [CaTS-Bench: Can Language Models Describe Time Series?](https://mlsanity.com/paper/2509.20823/) (2025)
- [TimeOmni-VL: Unified Models for Time Series Understanding and Generation](https://mlsanity.com/paper/2602.17149/) (2026)
- [Adaptive Time Series Reasoning via Segment Selection](https://mlsanity.com/paper/2602.18645/) (2026)
- [PATRA: Pattern-Aware Alignment and Balanced Reasoning for Time Series Question Answering](https://mlsanity.com/paper/2602.23161/) (2026)
- [TSQueryBench: LLM-as-a-Judge for Time Series Explanations](https://mlsanity.com/paper/2604.02118/) (2026)
- [SenTSR-Bench: Thinking with Injected Knowledge for Time-Series Reasoning](https://mlsanity.com/paper/2602.19455/) (2026)
- [FactoryBench: Evaluating Industrial Machine Understanding](https://mlsanity.com/paper/2605.07675/) (2026)

## Abstract (arXiv)

In recent years, time-series question answering (QA) systems have made significant progress. However, generating a correct answer does not show whether retaining the supplied numerical series improves task performance, nor whether the prediction is sensitive to changes in that input. While some systems provide rationales, answer accuracy also does not show whether their numerical claims are grounded in the supplied series or whether the stated inference is valid. In this work, we focus on evaluating four time-series QA systems: TimeOmni-1, ChatTS, TimeOmni-VL, and Time-MQA. First, for three systems with released evaluation data, we reproduce their reported results and compare the performance of the systems with their backbones. Then, we introduce a benchmark named COMMON-TSQA, which collects public evaluation datasets from existing time-series benchmarks and unifies their sample representation, task definitions, and answer schemas, while evaluating each system through its own interface under common evaluation criteria. The evaluation uses the original condition and six interventions while keeping the question and target fixed. Our analysis shows that aggregate performance alone can obscure how systems use numerical evidence. Similar task-level scores can arise despite substantial changes in individual predictions. Some interventions induce simple fallback behavior rather than preserved task ability. We also evaluate rationales for factual grounding, inference validity, and consistency with the final answer. We find that rationales often contain time-series claims unsupported by the input. Moreover, the rationale audit shows that agreement between a rationale and its final answer can coexist with incorrect numerical descriptions or invalid intermediate inferences.