mlsanityData to 14 Oct 2026

FACE: Evaluating Natural Language Generation with Fourier Analysis of Cross-Entropy

arXiv 2305.10307 ↗First posted 17 May 2023cs.CLcited by 6

Not on the map. Why?

Abstract (arXiv)

Measuring the distance between machine-produced and human language is a critical open problem. Inspired by empirical findings from psycholinguistics on the periodicity of entropy in language, we propose FACE, a set of metrics based on Fourier Analysis of the estimated Cross-Entropy of language, for measuring the similarity between model-generated and human-written languages. Based on an open-ended generation task and the experimental data from previous studies, we find that FACE can effectively identify the human-model gap, scales with model size, reflects the outcomes of different sampling methods for decoding, correlates well with other evaluation metrics and with human judgment scores.

Key facts. Cited by 6 papers in the map (data through 14 Oct 2026). Builds on 18 of its 53 references that are in the map. First posted on arXiv 17 May 2023.

Builds on

18 of 53 references.

References · 18

  1. 2019BERTScore: Evaluating Text Generation with BERTcited by 5,512
  2. 2022OPT: Open Pre-trained Transformer Language Modelscited by 3,512
  3. 2016Pointer Sentinel Mixture Modelscited by 3,096
  4. 2019The Curious Case of Neural Text Degenerationcited by 2,916
  5. 2021SimCSE: Simple Contrastive Learning of Sentence Embeddingscited by 2,173
  6. 2022BLOOM: A 176B-Parameter Open-Access Multilingual Language Modelcited by 1,984
  7. 2020BLEURT: Learning Robust Metrics for Text Generationcited by 1,203
  8. 2018Hierarchical Neural Story Generationcited by 1,193
  9. 2019Improved Precision and Recall Metric for Assessing Generative Modelscited by 944
  10. 2022Contrastive Decoding: Open-ended Text Generation as Optimizationcited by 609
  11. 2019Neural Text Generation with Unlikelihood Trainingcited by 503
  12. 2019GLTR: Statistical Detection and Visualization of Generated Textcited by 496
6 more
  1. 2018Texygen: A Benchmarking Platform for Text Generation Modelscited by 491
  2. 2021MAUVE: Measuring the Gap Between Neural Text and Human Text using Divergence Frontierscited by 470
  3. 2019MoverScore: Text Generation Evaluating with Contextualized Embeddings and Earth Mover Distancecited by 418
  4. 2018Assessing Generative Models via Precision and Recallcited by 406
  5. 2022A Contrastive Framework for Neural Text Generationcited by 225
  6. 2019Unifying Human and Statistical Evaluation for Natural Language Generationcited by 127

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2024: 1; 2025: 2; 2026: 3.

6 papers.

  1. 2024Detecting Subtle Differences between Human and Model Languages Using Spectrum of Relative Likelihoodcited by 24
  2. 2025Identifying the Periodicity of Information in Natural Languagecited by 4
  3. 2026LLMs Exhibit Significantly Lower Uncertainty in Creative Writing Than Professional Writerscited by 4
  4. 2025TempTest: Local Normalization Distortion and the Detection of Machine-generated Textcited by 3
  5. 2026WaveDetect: Robust Framework for Machine-Generated Text Detection via Wavelet Transformcited by 2
  6. 2026The Dynamics of Human and AI-Generated Language: How Semantics Fluctuates across Different Timescalescited by 0