Data to 14 Oct 2026

SPINE: SParse Interpretable Neural Embeddings

arXiv 1711.08792 ↗First posted 23 Nov 2017cs.CLcited by 55

Not on the map. Why?

Abstract (arXiv)

Prediction without justification has limited utility. Much of the success of neural models can be attributed to their ability to learn rich, dense and expressive representations. While these representations capture the underlying complexity and latent trends in the data, they are far from being interpretable. We propose a novel variant of denoising k-sparse autoencoders that generates highly efficient and interpretable distributed word representations (word embeddings), beginning with existing word representations from state-of-the-art methods like GloVe and word2vec. Through large scale human evaluation, we report that our resulting word embedddings are much more interpretable than the original GloVe and word2vec embeddings. Moreover, our embeddings outperform existing popular word embeddings on a diverse suite of benchmark downstream tasks.

Key facts. Cited by 55 papers in the map (data through 14 Oct 2026). First posted on arXiv 23 Nov 2017.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2021: 3; 2022: 10; 2023: 11; 2024: 14; 2025: 14; 2026: 3.

55 papers.

Sort:

By area:

2023Sparse Autoencoders Find Highly Interpretable Features in Language Modelscited by 1,420
2022From Anecdotal Evidence to Quantitative Evaluation Methods: A Systematic Review on Evaluating Explainable AIcited by 302
2022Toward Transparent AI: A Survey on Interpreting the Inner Structures of Deep Neural Networkscited by 129
2024Towards Principled Evaluations of Sparse Autoencoders for Interpretability and Controlcited by 86
2025A Survey on Sparse Autoencoders: Interpreting the Internal Mechanisms of Large Language Modelscited by 76
2024Identifying Functionally Important Features with End-to-End Sparse Dictionary Learningcited by 67
2022ProtoPFormer: Concentrating on Prototypical Parts in Vision Transformers for Interpretable Image Recognitioncited by 63
2025Projecting Assumptions: The Duality Between Sparse Autoencoders and Concept Geometrycited by 61
2022Visual Recognition with Deep Nearest Centroidscited by 59
2024Interpreting Attention Layer Outputs with Sparse Autoencoderscited by 43
2022On the Explainability of Natural Language Processing Deep Modelscited by 38
2024Dictionary Learning Improves Patch-Free Circuit Discovery in Mechanistic Interpretability: A Case Study on Othello-GPTcited by 29

38 more

2024Mechanistic?cited by 25
2025Interpretable and Testable Vision Features via Sparse Autoencoderscited by 25
2024Automatically Identifying Local and Global Circuits with Linear Computation Graphscited by 24
2021Interpreting Deep Learning Models in Natural Language Processing: A Reviewcited by 19
2023Discovering Universal Geometry in Embeddings with ICAcited by 19
2025A Comprehensive Survey on Self-Interpretable Neural Networkscited by 18
2025From superposition to sparse codes: interpretable representations in neural networkscited by 16
2022VICE: Variational Interpretable Concept Embeddingscited by 15
2023Tsetlin Machine Embedding: Representing Words Using Logical Expressionscited by 15
2022Semantic features of object concepts generated with GPT-3cited by 12
2023Sparsity-Guided Holistic Explanation for LLMs with Interpretable Inference-Time Interventioncited by 12
2024Encourage or Inhibit Monosemanticity? Revisit Monosemanticity from a Feature Decorrelation Perspectivecited by 10
2025Position: Text Embeddings Should Capture Implicit Semantics, Not Just Surface Meaningcited by 10
2025Re-Imagining Multimodal Instruction Tuning: A Representation Viewcited by 9
2023DINE: Dimensional Interpretability of Node Embeddingscited by 8
2022Interpreting Embedding Spaces by Conceptualizationcited by 6
2025Quantifying Structure in CLIP Embeddings: A Statistical Framework for Concept Interpretationcited by 6
2025Unboxing the Black Box: Mechanistic Interpretability for Algorithmic Understanding of Neural Networkscited by 6
2024Revisiting Cosine Similarity via Normalized ICA-transformed Embeddingscited by 5
2024Focus On This, Not That! Steering LLMs with Adaptive Feature Specificationcited by 5
2024Beyond Label Attention: Transparency in Language Models for Automated Medical Coding via Dictionary Learningcited by 5
2025Evaluating SAE interpretability without explanationscited by 5
2021Biomedical Interpretable Entity Representationscited by 4
2023Competence-Based Analysis of Language Modelscited by 4
2024DILA: Dictionary Label Attention for Mechanistic Interpretability in High-dimensional Multi-label Medical Coding Predictioncited by 4
2022AUTOLEX: An Automatic Framework for Linguistic Explorationcited by 3
2022Seeking Interpretability and Explainability in Binary Activated Neural Networkscited by 3
2023On the Relationship Between Interpretability and Explainability in Machine Learningcited by 3
2025Disentangling Linguistic Features with Dimension-Wise Analysis of Vector Embeddingscited by 2
2023Character-level Chinese Backpack Language Modelscited by 1
2024Disentangled and Self-Explainable Node Representation Learningcited by 1
2025Exploring Neural Ordinary Differential Equations as Interpretable Healthcare classifierscited by 1
2025Know-MRI: A Knowledge Mechanisms Revealer&Interpreter for Large Language Modelscited by 1
2021SEMIE: SEMantically Infused Embeddings with Enhanced Interpretability for Domain-specific Small Corpuscited by 0
2023A Scalable Space-efficient In-database Interpretability Framework for Embedding-based Semantic SQL Queriescited by 0
2023Curatr: A Platform for Semantic Analysis and Curation of Historical Literary Textscited by 0
2023Interpretable Neural Embeddings with Sparse Self-Representationcited by 0
2024Explainable by-design Audio Segmentation through Non-Negative Matrix Factorization and Probingcited by 0

50 most cited of 55 All 55 (JSON)