mlsanityData to 14 Oct 2026

FERA: Uncertainty-Aware Federated Reasoning for Large Language Models

arXiv 2605.10082 ↗First posted 11 May 2026cs.CL cs.LGcited by 1

On the map: LLM reasoning → Test-time compute and efficient thinking

Abstract (arXiv)

Large language models (LLMs) exhibit strong reasoning capabilities when guided by high-quality demonstrations, yet such data is often distributed across organizations that cannot centralize it due to regulatory, proprietary, or institutional constraints. We study federated reasoning, where a server improves multi-step reasoning by coordinating with heterogeneous clients holding private demonstrations, without centralized training or raw data sharing. The key challenge is that client reliability is query-dependent, while the server cannot inspect client data to determine which contributions are trustworthy. To address this, we propose Uncertainty-Aware Federated Reasoning (FERA), a training-free framework based on iterative server-client co-refinement. Across communication rounds, clients generate reasoning traces with lightweight uncertainty estimates, and the server synthesizes them into improved reasoning that is redistributed as context for the next round, progressively improving both server outputs and client-side reasoning. Within each round, Uncertainty-Aware Self-Critique Aggregation (UA-SCA) resolves conflicts among heterogeneous client traces through query-dependent trust weighting and structured cross-client verification. Rather than simply discarding low-quality traces, UA-SCA revises flawed reasoning steps to recover useful information. We provide theoretical guarantees showing that the proposed iterative protocol converges and that uncertainty-aware weighting accelerates convergence. Experiments on multiple reasoning benchmarks show that FERA consistently outperforms both federated training and training-free baselines, achieving progressively higher accuracy across rounds while maintaining communication and computational efficiency.

Key facts. Cited by 1 paper in the map (data through 14 Oct 2026). Placed in Test-time compute and efficient thinking, one of 17 directions in LLM reasoning. Builds on 63 of its 72 references that are in the map. First posted on arXiv 11 May 2026.

Builds on

63 of 72 references.

Closest ancestors · 5

Same direction, newest first

  1. 2025The Illusion of Thinking: Understanding the Strengths and Limitations of Reasoning Models via the Lens of Problem Complexity7 Jun
  2. 2025TokUR: Token-Level Uncertainty Estimation for Large Language Model Reasoning16 May
  3. 2025Dancing with Critiques: Enhancing LLM Reasoning with Stepwise Natural Language Self-Critique21 Mar
  4. 2023Tree of Thoughts: Deliberate Problem Solving with Large Language Models17 May
  5. 2023Federated Prompting and Chain-of-Thought Reasoning for Improving LLMs Answering27 Apr

Other references · 58

  1. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,399
  2. 2023GPT-4 Technical Reportcited by 18,396
  3. 2024The Llama 3 Herd of Modelscited by 14,715
  4. 2023Llama 2: Open Foundation and Fine-Tuned Chat Modelscited by 13,028
  5. 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,459
  6. 2021Training Verifiers to Solve Math Word Problemscited by 10,109
  7. 2019Sentence-BERT: Sentence Embeddings using Siamese BERT-Networkscited by 9,170
  8. 2025Qwen3 Technical Reportcited by 8,820
  9. 2022ReAct: Synergizing Reasoning and Acting in Language Modelscited by 8,695
  10. 2016Communication-Efficient Learning of Deep Networks from Decentralized Datacited by 7,669
  11. 2020Scaling Laws for Neural Language Modelscited by 6,586
  12. 2022Self-Consistency Improves Chain of Thought Reasoning in Language Modelscited by 5,708
38 more
  1. 2022Large Language Models are Zero-Shot Reasonerscited by 4,374
  2. 2023Reflexion: Language Agents with Verbal Reinforcement Learningcited by 4,276
  3. 2024MMLU-Pro: A More Robust and Challenging Multi-Task Language Understanding Benchmarkcited by 1,848
  4. 2023Improving Factuality and Reasoning in Language Models through Multiagent Debatecited by 1,784
  5. 2022Least-to-Most Prompting Enables Complex Reasoning in Large Language Modelscited by 1,407
  6. 2019Measuring the Effects of Non-Identical Data Distribution for Federated Visual Classificationcited by 851
  7. 2023Semantic Uncertainty: Linguistic Invariances for Uncertainty Estimation in Natural Language Generationcited by 819
  8. 2022What Can Transformers Learn In-Context? A Case Study of Simple Function Classescited by 715
  9. 2017Program Induction by Rationale Generation : Learning to Solve and Explain Algebraic Word Problemscited by 650
  10. 2023Plan-and-Solve Prompting: Improving Zero-Shot Chain-of-Thought Reasoning by Large Language Modelscited by 560
  11. 2023Trained Transformers Learn Linear Models In-Contextcited by 337
  12. 2023Towards Building the Federated GPT: Federated Instruction Tuningcited by 185
  13. 2023Navigate through Enigmatic Labyrinth A Survey of Chain of Thought Reasoning: Advances, Frontiers and Futurecited by 184
  14. 2023Shifting Attention to Relevance: Towards the Predictive Uncertainty Quantification of Free-Form Large Language Modelscited by 150
  15. 2023FederatedScope-LLM: A Comprehensive Package for Fine-tuning Large Language Models in Federated Learningcited by 149
  16. 2024FLoRA: Federated Fine-Tuning Large Language Models with Heterogeneous Low-Rank Adaptationscited by 146
  17. 2024OpenFedLLM: Training Large Language Models on Decentralized Private Data via Federated Learningcited by 128
  18. 2023Decomposing Uncertainty for Large Language Models through Input Clarification Ensemblingcited by 114
  19. 2023Cumulative Reasoning with Large Language Modelscited by 94
  20. 2023Answering Questions by Meta-Reasoning over Multiple Chains of Thoughtcited by 87
  21. 2024FedBiOT: LLM Local Fine-tuning in Federated Learning without Full Modelcited by 74
  22. 2023A Survey of Reasoning with Foundation Modelscited by 70
  23. 2023Federated Learning of Large Language Models with Parameter-Efficient Prompt Tuning and Adaptive Optimizationcited by 65
  24. 2023FATE-LLM: A Industrial Grade Federated Learning Framework for Large Language Modelscited by 63
  25. 2023Federated Full-Parameter Tuning of Billion-Sized Language Models with Communication Cost under 18 Kilobytescited by 55
  26. 2025Empowering LLMs with Logical Reasoning: A Comprehensive Surveycited by 51
  27. 2024The Future of Large Language Model Pre-training is Federatedcited by 30
  28. 2023Improving the Reliability of Large Language Models by Leveraging Uncertainty-Aware In-Context Learningcited by 29
  29. 2024Enhancing Reasoning Capabilities of LLMs via Principled Synthetic Logic Corpuscited by 29
  30. 2025Leaky Thoughts: Large Reasoning Models Are Not Private Thinkerscited by 28
  31. 2025Reinforce LLM Reasoning through Multi-Agent Reflectioncited by 27
  32. 2024Privacy Preserving Prompt Engineering: A Surveycited by 25
  33. 2025Can Textual Gradient Work in Federated Learning?cited by 15
  34. 2024Towards Federated RLHF with Aggregated Client Preference for LLMscited by 13
  35. 2024Bridging Model Heterogeneity in Federated Learning via Uncertainty-based Asymmetrical Reciprocity Learningcited by 11
  36. 2023Tunable Soft Prompts are Messengers in Federated Learningcited by 10
  37. 2024Federated In-Context LLM Agent Learningcited by 10
  38. 2024Unlocking the Power of LLM Uncertainty for Active In-Context Example Selectioncited by 9

50 most cited of 58 All 58 (JSON)

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 1.

1 paper.

  1. 2026Harness Handbook: Making Evolving Agent Harnesses Readable,Navigable, and Editablecited by 13

Similar, not cited

Close in wording, no citation either way

  1. 2026Federated Reasoning Distillation Framework with Model Learnability-Aware Data Allocation
  2. 2025FedCoT: Communication-Efficient Federated Reasoning Enhancement for Large Language Models
  3. 2026SELFDOUBT: Uncertainty Quantification for Reasoning LLMs via the Hedge-to-Verify Ratio
  4. 2026FedeKD: Energy-Based Gating for Robust Federated Knowledge Distillation under Heterogeneous Settings
  5. 2026Conflict-Aware Federated Fine-Tuning of Large Language Models with Mixture-of-Experts
  6. 2025A Comprehensive Survey on Trustworthiness in Reasoning with Large Language Models
  7. 2025Red Teaming Large Reasoning Models
  8. 2026Concordia: Self-Improving Synthetic Tables for Federated LLMs