mlsanityData to 7 Oct 2026

Aligning with Logic: Measuring, Evaluating and Improving Logical Preference Consistency in Large Language Models

arXiv 2410.02205 ↗First posted 3 Oct 2024cs.CL cs.AI cs.LOcited by 12

On the map: LLM alignment / RLHF / preference optimisation → Human values and AI feedback alignment

Abstract (arXiv)

Large Language Models (LLMs) are expected to be predictable and trustworthy to support reliable decision-making systems. Yet current LLMs often show inconsistencies in their judgments. In this work, we examine logical preference consistency as a foundational requirement for building more dependable LLM systems, ensuring stable and coherent decision-making while minimizing erratic or contradictory outputs. To quantify the logical preference consistency, we propose a universal evaluation framework based on three fundamental properties: transitivity, commutativity and negation invariance. Through extensive experimentation across diverse LLMs, we demonstrate that these properties serve as strong indicators of judgment robustness. Furthermore, we introduce a data refinement and augmentation technique, REPAIR, that enhances logical consistency while maintaining alignment with human preferences. Finally, we show that improving consistency leads to better performance in LLM-driven logic-based algorithms, reinforcing stability and coherence in decision-making systems.

Key facts. Cited by 12 papers in the map (data through 7 Oct 2026). Placed in Human values and AI feedback alignment, one of 10 directions in LLM alignment / RLHF / preference optimisation. Builds on 19 of its 71 references that are in the map. First posted on arXiv 3 Oct 2024.

Builds on

19 of 71 references.

Closest ancestors · 1

Same direction, newest first

  1. 2024Fairer Preferences Elicit Improved Human-Aligned Large Language Model Judgments17 Jun

Other references · 18

  1. 2023GPT-4 Technical Reportcited by 18,132
  2. 2022Training language models to follow instructions with human feedbackcited by 15,712
  3. 2024The Llama 3 Herd of Modelscited by 14,192
  4. 2023Llama 2: Open Foundation and Fine-Tuned Chat Modelscited by 12,865
  5. 2022PaLM: Scaling Language Modeling with Pathwayscited by 5,304
  6. 2022Training a Helpful and Harmless Assistant with Reinforcement Learning from Human Feedbackcited by 3,634
  7. 2023Mistral 7Bcited by 3,065
  8. 2024Gemma 2: Improving Open Language Models at a Practical Sizecited by 1,928
  9. 2024Phi-3 Technical Report: A Highly Capable Language Model Locally on Your Phonecited by 1,883
  10. 2023PaLM 2 Technical Reportcited by 1,154
  11. 2024DeepSeek-V2: A Strong, Economical, and Efficient Mixture-of-Experts Language Modelcited by 1,093
  12. 2023Gemini: A Family of Highly Capable Multimodal Modelscited by 810
6 more
  1. 2023Siren's Song in the AI Ocean: A Survey on Hallucination in Large Language Modelscited by 695
  2. 2023An Empirical Study of Catastrophic Forgetting in Large Language Models During Continual Fine-tuningcited by 622
  3. 2023Zephyr: Direct Distillation of LM Alignmentcited by 498
  4. 2024Secrets of RLHF in Large Language Models Part II: Reward Modelingcited by 170
  5. 2023Physics of Language Models: Part 3.2, Knowledge Manipulationcited by 142
  6. 2024LLM as a Complementary Optimizer to Gradient Descent: A Case Study in Prompt Tuningcited by 5

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 5; 2026: 7.

12 papers.

  1. 2025Logical Reasoning in Large Language Models: A Surveycited by 52
  2. 2025Consistently Simulating Human Personas with Multi-Turn Reinforcement Learningcited by 35
  3. 2025Consistency in Language Models: Current Landscape, Challenges, and Future Directionscited by 14
  4. 2026Who can we trust? LLM-as-a-jury for Comparative Assessmentcited by 7
  5. 2025LISTEN to Your Preferences: An LLM Framework for Multi-Objective Selectioncited by 2
  6. 2026Discourse-Aware Dual-Track Streaming Response for Low-Latency Spoken Dialogue Systemscited by 2
  7. 2025Voting with the Graph: Stable RLAIF via Topological Consistency Maximizationcited by 1
  8. 2026PAVE: Premise-Aware Validation and Editing for Retrieval-Augmented LLMscited by 1
  9. 2026Grading the Unspoken: Evaluating Tacit Reasoning in Quantum Field Theory and String Theory with LLMscited by 1
  10. 2026Reasoning Capabilities of Large Language Models. Lessons Learned from General Game Playingcited by 0
  11. 2026Jagged Judges: Epistemic Stability Under Perturbation, Pressure, and Persistencecited by 0
  12. 2026LLM-Derived Preference Judgments Are Not Self-Consistentcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025Probability-Consistent Preference Optimization for Enhanced LLM Reasoning
  2. 2026Quantifying Consistency in LLM Logical Reasoning via Structural Uncertainty
  3. 2025Alignment Revisited: Are Large Language Models Consistent in Stated and Revealed Preferences?
  4. 2024ContraSolver: Self-Alignment of Language Models by Resolving Internal Preference Contradictions
  5. 2024Varying Shades of Wrong: Aligning LLMs with Wrong Answers Only
  6. 2026Preference Reasoning under Indeterminacy in Large Language Models
  7. 2026When Specifications Conflict: A Symmetry-Based Framework for Measuring LLM Preferences
  8. 2024Semantic Self-Consistency: Enhancing Language Model Reasoning via Semantic Weighting