mlsanityData to 7 Oct 2026

Regression-aware Inference with LLMs

arXiv 2403.04182 ↗First posted 7 Mar 2024cs.CL cs.AIcited by 13

Not on the map. Why?

Abstract (arXiv)

Large language models (LLMs) have shown strong results on a range of applications, including regression and scoring tasks. Typically, one obtains outputs from an LLM via autoregressive sampling from the model's output distribution. We show that this inference strategy can be sub-optimal for common regression and scoring evaluation metrics. As a remedy, we build on prior work on Minimum Bayes Risk decoding, and propose alternate inference strategies that estimate the Bayes-optimal solution for regression and scoring metrics in closed-form from sampled responses. We show that our proposal significantly improves over baselines across datasets and models.

Key facts. Cited by 13 papers in the map (data through 7 Oct 2026). Builds on 17 of its 34 references that are in the map. First posted on arXiv 7 Mar 2024.

Builds on

17 of 34 references.

References · 17

  1. 2022Self-Consistency Improves Chain of Thought Reasoning in Language Modelscited by 5,601
  2. 2021Finetuned Language Models Are Zero-Shot Learnerscited by 3,747
  3. 2017TriviaQA: A Large Scale Distantly Supervised Challenge Dataset for Reading Comprehensioncited by 2,895
  4. 2018Hierarchical Neural Story Generationcited by 1,182
  5. 2017SemEval-2017 Task 1: Semantic Textual Similarity - Multilingual and Cross-lingual Focused Evaluationcited by 998
  6. 2023Large Language Modelscited by 774
  7. 2022Large Language Models with Controllable Working Memorycited by 169
  8. 2020Is MAP Decoding All You Need? The Inadequacy of the Mode in Neural Machine Translationcited by 137
  9. 2023Goat: Fine-tuned LLaMA Outperforms GPT-4 on Arithmetic Taskscited by 84
  10. 2023GPT Can Solve Mathematical Problems Without a Calculatorcited by 68
  11. 2023Epsilon Sampling Rocks: Investigating Sampling Strategies for Minimum Bayes Risk Decoding for Machine Translationcited by 57
  12. 2022Follow the Wisdom of the Crowd: Effective Text Generation via Minimum Bayes Risk Decodingcited by 54
5 more
  1. 2023It's MBR All the Way Down: Modern Generation Techniques Through the Lens of Minimum Bayes Riskcited by 49
  2. 2023Faster Minimum Bayes Risk Decoding with Confidence-based Pruningcited by 34
  3. 2023MBR and QE Finetuning: Training-time Distillation of the Best and Most Expensive Decoding Methodscited by 33
  4. 2023Stealing the Decoding Algorithms of Language Modelscited by 19
  5. 2022DC-MBR: Distributional Cooling for Minimum Bayesian Risk Decodingcited by 4

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 6; 2026: 7.

13 papers.

Sort:

By area:

  1. 2025Decoding-based Regressioncited by 11
  2. 2025Uncertainty-Aware Decoding with Minimum Bayes Riskcited by 10
  3. 2025Quantile Regression with Large Language Models for Price Predictioncited by 9
  4. 2026VERTIGO: Visual Preference Optimization for Cinematic Camera Trajectory Generationcited by 3
  5. 2026NumericJev: Jev-like LLM Numerical Decoding with Multiway Decision Treescited by 3
  6. 2025Rethinking the Capability of Fine-Tuned Language Models for Automated Vulnerability Repaircited by 2
  7. 2026Task-Awareness Improves LLM Generations and Uncertaintycited by 2
  8. 2025Beyond Token-level Supervision: Unlocking the Potential of Decoding-based Regression via Reinforcement Learningcited by 1
  9. 2026REAL: Regression-Aware Reinforcement Learning for LLM-as-a-Judgecited by 1
  10. 2026RELISH: LLM REgression with a Latent Iterative State Headcited by 1
  11. 2026Reinforcement Learning over Predictive Distributions for LLM Regressioncited by 1
  12. 2025KidsArtBench: Multi-Dimensional Children's Art Evaluation with Attribute-Aware MLLMscited by 0
1 more
  1. 2026Task-Aware Calibration: Provably Optimal Decoding in LLMscited by 0