mlsanityData to 7 Oct 2026

Out-of-Distribution Detection using Synthetic Data Generation

arXiv 2502.03323 ↗First posted 5 Feb 2025cs.CL cs.AI cs.LGcited by 4

On the map: Synthetic data and self-improvement for LLMs → LLM synthetic data generation

Abstract (arXiv)

Distinguishing in- and out-of-distribution (OOD) inputs is crucial for reliable deployment of classification systems. However, OOD data is typically unavailable or difficult to collect, posing a significant challenge for accurate OOD detection. In this work, we present a method that harnesses the generative capabilities of Large Language Models (LLMs) to create high-quality synthetic OOD proxies, eliminating the dependency on any external OOD data source. We study the efficacy of our method on classical text classification tasks such as toxicity detection and sentiment classification as well as classification tasks arising in LLM development and deployment, such as training a reward model for RLHF and detecting misaligned generations. Extensive experiments on nine InD-OOD dataset pairs and various model sizes show that our approach dramatically lowers false positive rates (achieving a perfect zero in some cases) while maintaining high accuracy on in-distribution tasks, outperforming baseline methods by a significant margin.

Key facts. Cited by 4 papers in the map (data through 7 Oct 2026). Placed in LLM synthetic data generation, one of 13 directions in Synthetic data and self-improvement for LLMs. Builds on 86 of its 100 references that are in the map. First posted on arXiv 5 Feb 2025.

Builds on

86 of 100 references.

Closest ancestors · 1

Same direction, newest first

  1. 2024On the Diversity of Synthetic Data and its Impact on Training Large Language Models19 Oct

Other references · 85

  1. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
  2. 2022Training language models to follow instructions with human feedbackcited by 15,712
  3. 2021LoRA: Low-Rank Adaptation of Large Language Modelscited by 14,084
  4. 2023LLaMA: Open and Efficient Foundation Language Modelscited by 13,958
  5. 2021Training Verifiers to Solve Math Word Problemscited by 9,743
  6. 2019Sentence-BERT: Sentence Embeddings using Siamese BERT-Networkscited by 9,049
  7. 2015Dropout as a Bayesian Approximation: Representing Model Uncertainty in Deep Learningcited by 4,416
  8. 2016Simple and Scalable Predictive Uncertainty Estimation using Deep Ensemblescited by 3,888
  9. 2017Deep reinforcement learning from human preferencescited by 3,846
  10. 2021Program Synthesis with Large Language Modelscited by 3,509
  11. 2018UMAP: Uniform Manifold Approximation and Projection for Dimension Reductioncited by 2,841
  12. 2022Self-Instruct: Aligning Language Models with Self-Generated Instructionscited by 2,674
38 more
  1. 2016A Baseline for Detecting Misclassified and Out-of-Distribution Examples in Neural Networkscited by 2,362
  2. 2023Pythia: A Suite for Analyzing Large Language Models Across Training and Scalingcited by 1,734
  3. 2018A Simple Unified Framework for Detecting Out-of-Distribution Samples and Adversarial Attackscited by 1,472
  4. 2017Enhancing The Reliability of Out-of-distribution Image Detection in Neural Networkscited by 1,325
  5. 2020Energy-based Out-of-distribution Detectioncited by 1,296
  6. 2024The FineWeb Datasets: Decanting the Web for the Finest Text Data at Scalecited by 1,104
  7. 2022Scaling Laws for Reward Model Overoptimizationcited by 1,056
  8. 2018Deep Anomaly Detection with Outlier Exposurecited by 939
  9. 2023BeaverTails: Towards Improved Safety Alignment of LLM via a Human-Preference Datasetcited by 828
  10. 2017Selective Classification for Deep Neural Networkscited by 799
  11. 2021Generalized Out-of-Distribution Detection: A Surveycited by 796
  12. 2014Deep Neural Networks are Easily Fooled: High Confidence Predictions for Unrecognizable Imagescited by 774
  13. 2018Predictive Uncertainty Estimation via Prior Networkscited by 555
  14. 2015Towards Open Set Deep Networkscited by 533
  15. 2021ReAct: Out-of-distribution Detection With Rectified Activationscited by 486
  16. 2024RewardBench: Evaluating Reward Models for Language Modelingcited by 480
  17. 2020CSI: Novelty Detection via Contrastive Learning on Distributionally Shifted Instancescited by 441
  18. 2022ToxiGen: A Large-Scale Machine-Generated Dataset for Adversarial and Implicit Hate Speech Detectioncited by 419
  19. 2019Nuanced Metrics for Measuring Unintended Bias with Real Data for Text Classificationcited by 396
  20. 2017Training Confidence-calibrated Classifiers for Detecting Out-of-Distribution Samplescited by 389
  21. 2024DataComp-LM: In search of the next generation of training sets for language modelscited by 381
  22. 2020BatchEnsemble: An Alternative Approach to Efficient Ensemble and Lifelong Learningcited by 368
  23. 2019SelectiveNet: A Deep Neural Network with an Integrated Reject Optioncited by 333
  24. 2021Exploring the Limits of Out-of-Distribution Detectioncited by 283
  25. 2022OpenOOD: Benchmarking Generalized Out-of-Distribution Detectioncited by 267
  26. 2022Mitigating Neural Network Overconfidence with Logit Normalizationcited by 251
  27. 2018Using Trusted Data to Train Deep Networks on Labels Corrupted by Severe Noisecited by 216
  28. 2022Extremely Simple Activation Shaping for Out-of-Distribution Detectioncited by 212
  29. 2021A Simple Fix to Mahalanobis Distance for Improving Near-OOD Detectioncited by 206
  30. 2020Contrastive Training for Improved Out-of-Distribution Detectioncited by 186
  31. 2023AnomalyGPT: Detecting Industrial Anomalies Using Large Vision-Language Modelscited by 169
  32. 2021DICE: Leveraging Sparsification for Out-of-Distribution Detectioncited by 166
  33. 2023Does Synthetic Data Generation of LLMs Help Clinical Text Mining?cited by 151
  34. 2021PixMix: Dreamlike Pictures Comprehensively Improve Safety Measurescited by 131
  35. 2022How to Exploit Hyperspherical Embeddings for Out-of-Distribution Detection?cited by 128
  36. 2022Is Out-of-Distribution Detection Learnable?cited by 109
  37. 2022POEM: Out-of-Distribution Detection with Posterior Samplingcited by 108
  38. 2021Semantically Coherent Out-of-Distribution Detectioncited by 107

50 most cited of 85 All 85 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 3; 2026: 1.

4 papers.

  1. 2025Graph Synthetic Out-of-Distribution Exposure with Large Language Modelscited by 5
  2. 2025GOOD: Training-Free Guided Diffusion Sampling for Out-of-Distribution Detectioncited by 5
  3. 2026Trust The Typicalcited by 3
  4. 2025Medix: Out-of-Distribution Detection from Unlabeled Wild Data via Robust Gradient Statisticscited by 2

Similar, not cited

Close in wording, no citation either way

  1. 2023Generating Faithful Synthetic Data with Large Language Models: A Case Study in Computational Social Science
  2. 2024Envisioning Outlier Exposure by Large Language Models for Out-of-Distribution Detection
  3. 2023Synthetic Data Generation with Large Language Models for Text Classification: Potential and Limitations
  4. 2026Benchmarking and Improving Monitors for Out-Of-Distribution Alignment Failure in LLMs
  5. 2025Improving the OOD Performance of Closed-Source LLMs on NLI Through Strategic Data Selection
  6. 2025Less is More: Adaptive Coverage for Synthetic Training Data
  7. 2026Debiased Negative Mining Improves Out-of-distribution Detection with Pre-trained Vision-Language Models
  8. 2025Synthetic Data Generation Using Large Language Models: Advances in Text and Code