mlsanityData to 7 Oct 2026

QLENS: Towards A Quantum Perspective of Language Transformers

arXiv 2510.11963 ↗First posted 13 Oct 2025cs.LGcited by 0

Not on the map. Why?

Abstract (arXiv)

In natural language processing, current methods for understanding Transformers are successful at identifying intermediate predictions during a model's inference. However, these approaches function as limited diagnostic checkpoints, lacking a mathematical framework for mechanistically modeling how each layer facilitates transitions between these evolving states. This interpretability gap and past successes of interdisciplinary outlooks inspire us to turn to physics in search of a descriptive mathematical framework for Transformers. We observe that language models are intrinsically probabilistic, an attribute that is echoed in the core postulates of quantum mechanics. This parallel inspires us to translate insights from this discipline to that of natural language processing. Towards this objective, we propose QLENS a novel attempt to develop a physics-based perspective on the Transformer generation process. Under QLENS, a Transformer is studied by converting its latent activations into a state vector in a Hilbert space derived from the model's output units. This state subsequently evolves through hidden layers - reformulated as unitary operators and analogously defined Hamiltonians - during inference. The model's final probability distribution is obtained by applying the Born rule to the end state using a specific measurement operator. To demonstrate QLENS's potential, we conduct a proof-of-concept by probing a toy Transformer to investigate the influence of individual layers in a model's prediction trajectory. We present our work as a foundation for cross-domain insights to be leveraged towards a broader understanding of Transformers.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 25 of its 57 references that are in the map. First posted on arXiv 13 Oct 2025.

Builds on

25 of 57 references.

References · 25

  1. 2017Attention Is All You Needcited by 59,932
  2. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  3. 2020Language Models are Few-Shot Learnerscited by 32,199
  4. 2024The Llama 3 Herd of Modelscited by 14,192
  5. 2013Efficient Estimation of Word Representations in Vector Spacecited by 4,346
  6. 2016Gaussian Error Linear Units (GELUs)cited by 3,431
  7. 2018Self-Attentive Sequential Recommendationcited by 1,169
  8. 2022GPT-NeoX-20B: An Open-Source Autoregressive Language Modelcited by 711
  9. 2023Towards Automated Circuit Discovery for Mechanistic Interpretabilitycited by 700
  10. 2023Eliciting Latent Predictions from Transformers with the Tuned Lenscited by 579
  11. 2023TinyStories: How Small Can Language Models Be and Still Speak Coherent English?cited by 425
  12. 2023Explainability for Large Language Models: A Surveycited by 331
13 more
  1. 2017Searching for Activation Functionscited by 158
  2. 2024The Remarkable Robustness of LLMs: Stages of Inference?cited by 138
  3. 2018What Is One Grain of Sand in the Desert? Analyzing Individual Neurons in Deep NLP Modelscited by 124
  4. 2020On the Effect of Dropping Layers of Pre-trained Transformer Modelscited by 118
  5. 2017Residual Connections Encourage Iterative Inferencecited by 74
  6. 2022Quantum Vision Transformerscited by 48
  7. 2022Quantum Self-Attention Neural Networks for Text Classificationcited by 42
  8. 2016SentiHood: Targeted Aspect Based Sentiment Analysis Dataset for Urban Neighbourhoodscited by 32
  9. 2024An Investigation of Neuron Activation as a Unified Lens to Explain Chain-of-Thought Eliciting Arithmetic Reasoning of LLMscited by 20
  10. 2024Quantum Mixed-State Self-Attention Networkcited by 11
  11. 2025Neural Thermodynamic Laws for Large Language Model Trainingcited by 10
  12. 2025Quantum-Enhanced LLM Efficient Fine Tuningcited by 6
  13. 2025On the Effect of Uncertainty on Layer-wise Inference Dynamicscited by 1

Built on by

Not cited yet.