mlsanityData to 14 Oct 2026

Reinforcement Learning with Comparative Evidence for Social Intelligence

arXiv 2610.04072 ↗First posted 2 Oct 2026cs.AIcited by 0

On the map: LLM reasoning → Reinforcement learning for reasoning

Abstract (arXiv)

Developing socially intelligent AI remains heavily dependent on human-annotated data, limiting the scale and breadth of social understanding models can acquire. Methods that derive training signals from unlabeled data offer a path beyond this dependence, but social predictions lack the verification oracles available in mathematics and coding. Moreover, core social targets such as affect, intent, preference, and pragmatic meaning are often ambiguous. The same behavior can support multiple plausible interpretations, making it difficult to verify which is best supported. To address this challenge, we introduce Reinforcement Learning with Comparative Evidence (RLCE), a reinforcement learning method that learns social understanding from unlabeled training data without constructing rewards from ground-truth annotations. Given distinct answers in a rollout group, RLCE constructs evidence tests that identify observable evidence favoring an answer over another, validates these tests against the input sample, and aggregates test outcomes to determine the best-supported interpretation. Tests are regenerated as the policy produces new answers, enabling them to evolve with the policy. Across four benchmarks spanning affect, pragmatics, communicative intent, and preference, RLCE attains the strongest performance among seven methods that use no ground-truth training labels for rewards, including consensus, policy LLM-judge verification, multimodal co-evolution, and rubric-based rewards. Gains over the strongest baseline reach up to +18.93 points. Analyses further show that RLCE exhibits a larger share of reward variation between correct and incorrect predictions than compared rubric methods, can overturn erroneous policy-derived preferences, and benefits from pairwise test construction, compositional test aggregation, and on-policy test evolution.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in Reinforcement learning for reasoning, one of 17 directions in LLM reasoning. Builds on 28 of its 35 references that are in the map. First posted on arXiv 2 Oct 2026.

Builds on

28 of 35 references.

Closest ancestors · 7

Same direction, newest first

  1. 2026SERPO: Self-Evolving Rubric Policy Optimization for Open-Ended Test-Time Reinforcement Learning29 Jul
  2. 2025DR Tulu: Reinforcement Learning with Evolving Rubrics for Deep Research24 Nov
  3. 2025RLAC: Reinforcement Learning with Adversarial Critic for Free-Form Generation Tasks3 Nov
  4. 2025Rubrics as Rewards: Reinforcement Learning Beyond Verifiable Domains23 Jul
  5. 2025Absolute Zero: Reinforced Self-play Reasoning with Zero Data6 May
  6. 2025TTRL: Test-Time Reinforcement Learning22 Apr
  7. 2022Constitutional AI: Harmlessness from AI Feedback15 Dec

Other references · 21

  1. 2024DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Modelscited by 9,379
  2. 2023Judging LLM-as-a-Judge with MT-Bench and Chatbot Arenacited by 8,626
  3. 2022Self-Consistency Improves Chain of Thought Reasoning in Language Modelscited by 5,708
  4. 2021Finetuned Language Models Are Zero-Shot Learnerscited by 3,780
  5. 2023G-Eval: NLG Evaluation using GPT-4 with Better Human Alignmentcited by 2,276
  6. 2021Cross-Task Generalization via Natural Language Crowdsourcing Instructionscited by 699
  7. 2024Self-Rewarding Language Modelscited by 681
  8. 2019Pseudo-Labeling and Confirmation Bias in Deep Semi-Supervised Learningcited by 510
  9. 2023Universal Self-Consistency for Large Language Model Generationcited by 203
  10. 2025Do LLMs Recognize Your Preferences? Evaluating Personalized Preference Following in LLMscited by 148
  11. 2019UR-FUNNY: A Multimodal Language Dataset for Understanding Humorcited by 132
  12. 2024Uncertainty-Based Abstention in LLMs Improves Safety and Reduces Hallucinationscited by 54
9 more
  1. 2025VisPlay: Self-Evolving Vision-Language Models from Imagescited by 47
  2. 2025Social Genome: Grounded Social Reasoning Abilities of Multimodal Modelscited by 26
  3. 2024Affective Computing in the Era of Large Language Models: A Survey from the NLP Perspectivecited by 25
  4. 2024Advancing Social Intelligence in AI Agents: Technical Challenges and Open Questionscited by 22
  5. 2026EvoRubric: Self-Evolving Rubric-Driven RL for Open-Ended Generationcited by 11
  6. 2025Human Behavior Atlas: Benchmarking Unified Psychological and Social Behavior Understandingcited by 6
  7. 2026Video-Zero: Self-Evolution Video Understandingcited by 4
  8. 2026DeltaRubric: Generative Multimodal Reward Modeling via Joint Planning and Verificationcited by 3
  9. 2026OmniSapiens: A Foundation Model for Social Behavior Processing via Heterogeneity-Aware Relative Policy Optimizationcited by 1

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2026Social-R1: Towards Human-like Social Reasoning in LLMs
  2. 2025Do Theory of Mind Benchmarks Need Explicit Human-like Reasoning in Language Models?
  3. 2025Sotopia-RL: Reward Design for Social Intelligence
  4. 2026Reinforcement Learning Towards Broadly and Persistently Beneficial Models
  5. 2025Statistical Inference in Reinforcement Learning: A Selective Survey
  6. 2026RLHF May Not Reflect Genuine Preferences
  7. 2025TimeHC-RL: Temporal-aware Hierarchical Cognitive Reinforcement Learning for Enhancing LLMs' Social Intelligence
  8. 2025On the Interplay of Pre-Training, Mid-Training, and RL on Reasoning Language Models