mlsanityData to 7 Oct 2026

Inspection and Control of Self-Generated-Text Recognition Ability in Llama3-8b-Instruct

arXiv 2410.02064 ↗First posted 2 Oct 2024cs.LG cs.AI cs.CLcited by 15

On the map: Synthetic data and self-improvement for LLMs → Self-rewarding alignment and self-training

Abstract (arXiv)

It has been reported that LLMs can recognize their own writing. As this has potential implications for AI safety, yet is relatively understudied, we investigate the phenomenon, seeking to establish whether it robustly occurs at the behavioral level, how the observed behavior is achieved, and whether it can be controlled. First, we find that the Llama3-8b-Instruct chat model - but not the base Llama3-8b model - can reliably distinguish its own outputs from those of humans, and present evidence that the chat model is likely using its experience with its own outputs, acquired during post-training, to succeed at the writing recognition task. Second, we identify a vector in the residual stream of the model that is differentially activated when the model makes a correct self-written-text recognition judgment, show that the vector activates in response to information relevant to self-authorship, present evidence that the vector is related to the concept of "self" in the model, and demonstrate that the vector is causally related to the model's ability to perceive and assert self-authorship. Finally, we show that the vector can be used to control both the model's behavior and its perception, steering the model to claim or disclaim authorship by applying the vector to the model's output as it generates it, and steering the model to believe or disbelieve it wrote arbitrary texts by applying the vector to them as the model reads them.

Key facts. Cited by 15 papers in the map (data through 7 Oct 2026). Placed in Self-rewarding alignment and self-training, one of 13 directions in Synthetic data and self-improvement for LLMs. Builds on 5 of its 14 references that are in the map. First posted on arXiv 2 Oct 2024.

Builds on

5 of 14 references.

References · 5

  1. 2020Language Models are Few-Shot Learnerscited by 32,199
  2. 2018Don't Give Me the Details, Just the Summary! Topic-Aware Convolutional Neural Networks for Extreme Summarizationcited by 1,308
  3. 2023Representation Engineering: A Top-Down Approach to AI Transparencycited by 1,239
  4. 2015Teaching Machines to Read and Comprehendcited by 1,039
  5. 2023Eliciting Latent Predictions from Transformers with the Tuned Lenscited by 579

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 3; 2026: 12.

15 papers.

Sort:

By area:

  1. 2025Know Thyself? On the Incapability and Implications of AI Self-Recognitioncited by 7
  2. 2026Are LLM Evaluators Really Narcissists? Sanity Checking Self-Preference Evaluationscited by 5
  3. 2026ContextEcho: A Benchmark for Persona Drift in Long Agentic-Coding Sessionscited by 5
  4. 2025Mitigating Many-Shot Jailbreakingcited by 3
  5. 2025Breaking the Mirror: Activation-Based Mitigation of Self-Preference in LLM Evaluatorscited by 3
  6. 2026From Simulation to Enaction: Post-trained language models recognize and react to their own generationscited by 3
  7. 2026Evaluating Large Language Model Raters for German Open-Response Clinical Questions: A Physician-Annotated Benchmark Study of Agreement, Evaluator Bias, and Abstentioncited by 2
  8. 2026Self-Recognition Finetuning can Prevent and Reverse Emergent Misalignmentcited by 1
  9. 2026Self-Generated Text Recognition: Quality Heuristics, Cross-Task Transfer, and Downstream Bias in LLM Evaluationcited by 1
  10. 2026Robust Uncertainty Quantification for Factual Generation of Large Language Modelscited by 0
  11. 2026The Assistant as a Privileged Persona: A canonical reference in cross-persona self-recognitioncited by 0
  12. 2026LLM Self-Recognition: Steering and Retrieving Activation Signaturescited by 0
3 more
  1. 2026Telescope: Improving Zero Shot Detection of LLM Generated Content By Measuring Token Repetition Probabilitycited by 0
  2. 2026Strangers to Themselves: What Language Models Say About Themselves Is Genericcited by 0
  3. 2026Style, Not Self: Surface Cues Explain Zero-Shot Code Attribution by Large Language Modelscited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2024LLM Evaluators Recognize and Favor Their Own Generations
  2. 2025Prototypical Human-AI Collaboration Behaviors from LLM-Assisted Writing in the Wild
  3. 2023PersonaLLM: Investigating the Ability of Large Language Models to Express Personality Traits
  4. 2024SelfIE: Self-Interpretation of Large Language Model Embeddings
  5. 2024Self-Judge: Selective Instruction Following with Alignment Self-Evaluation
  6. 2026STEMMA: An Adversarial Multi-Agent Framework for Evaluating Self-Identity Consistency in LLMs
  7. 2025NeuroFaith: Evaluating LLM Self-Explanation Faithfulness via Internal Representation Alignment
  8. 2023Chat Vector: A Simple Approach to Equip LLMs with Instruction Following and Model Alignment in New Languages