# Synthetic Contrastive Reasoning for Multi-Table Q&A

arXiv: https://arxiv.org/abs/2606.05382 · HTML: https://mlsanity.com/paper/2606.05382/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Placed in Knowledge- and tool-augmented reasoning, one of 17 directions in LLM reasoning.
- Builds on 15 of its 33 references that are in the map.
- First posted on arXiv 3 Jun 2026.

## Placed in

- LLM reasoning / [Knowledge- and tool-augmented reasoning](https://mlsanity.com/lr/knowledge-and-tool-augmented-reasoning/)

## Builds on (most cited first, 15 of 15)

- [LLaMA: Open and Efficient Foundation Language Models](https://mlsanity.com/paper/2302.13971/) (2023)
- [GPT-4o System Card](https://mlsanity.com/paper/2410.21276/) (2024)
- [Qwen Technical Report](https://mlsanity.com/paper/2309.16609/) (2023)
- [Spider: A Large-Scale Human-Labeled Dataset for Complex and Cross-Domain Semantic Parsing and Text-to-SQL Task](https://mlsanity.com/paper/1809.08887/) (2018)
- [TaBERT: Pretraining for Joint Understanding of Textual and Tabular Data](https://mlsanity.com/paper/2005.08314/) (2020)
- [Contrastive Preference Optimization: Pushing the Boundaries of LLM Performance in Machine Translation](https://mlsanity.com/paper/2401.08417/) (2024)
- [Direct Preference Optimization with an Offset](https://mlsanity.com/paper/2402.10571/) (2024)
- [Contrastive Preference Learning: Learning from Human Feedback without RL](https://mlsanity.com/paper/2310.13639/) (2023)
- [TableGPT2: A Large Multimodal Model with Tabular Data Integration](https://mlsanity.com/paper/2411.02059/) (2024)
- [Contrastive Chain-of-Thought Prompting](https://mlsanity.com/paper/2311.09277/) (2023)
- [MultiTabQA: Generating Tabular Answers for Multi-Table Question Answering](https://mlsanity.com/paper/2305.12820/) (2023)
- [Table-R1: Inference-Time Scaling for Table Reasoning](https://mlsanity.com/paper/2505.23621/) (2025)
- [Reasoning-Table: Exploring Reinforcement Learning for Table Reasoning](https://mlsanity.com/paper/2506.01710/) (2025)
- [TQA-Bench: Evaluating LLMs for Multi-Table Question Answering](https://mlsanity.com/paper/2411.19504/) (2024)
- [MMTU: A Massive Multi-Task Table Understanding and Reasoning Benchmark](https://mlsanity.com/paper/2506.05587/) (2025)

## Abstract (arXiv)

Multi-table question answering requires models to retrieve relevant evidence, link schemas, and perform compositional reasoning across relational tables. Existing multi-table Q&A resources typically provide questions and final answers but lack reasoning supervision that explains how answers are derived. To address this gap, we construct a synthetic contrastive reasoning-trace dataset for MMQA by generating validated positive traces and plausible negative traces with heterogeneous LLMs. We then use the resulting preference pairs to fine-tune open-weight LLMs with Contrastive Preference Optimization (CPO). Across Qwen3-14B, Mistral-8B, and Llama-3.1-8B, CPO achieves absolute average improvements over Q&A supervised fine-tuning ranging from 9.7%-16.3%, with gains up to 21 percentage points on MMQA. Ablations show that heterogeneous positive and negative trace generators strengthen the contrastive signal, and automated as well as human evaluations indicate that the generated pairs are largely faithful, coherent, and meaningfully contrastive.