mlsanityData to 14 Oct 2026

Sense Representations Are Inducible Interfaces

arXiv 2605.28669 ↗First posted 27 May 2026cs.CL cs.AIcited by 0

Not on the map. Why?

Abstract (arXiv)

Sense representations (explicit, per-token meaning decompositions) are useful for disambiguation, steering, and cross-lingual alignment, but existing approaches require models to be pretrained with sense structure baked in. We introduce ACROS, which induces an explicit sense pathway into a frozen pretrained decoder LM through a gated residual addition. On SmolLM2-360M, ACROS preserves base LM quality while supporting three uses of the same induced variables: zero-shot word-sense disambiguation (64.95 F1 on Raganato ALL, competitive with the WordNet first-sense heuristic), low-KL lexical steering across 5,161 CoInCo cases where a simple non-oracle proxy recovers about 90% of positive shifts, and SENSIA cross-lingual adaptation to four languages (mean R@1 0.988, target FLORES PPL 7.94). ACROS makes sense representations an inducible interface for ordinary pretrained LMs.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 18 of its 31 references that are in the map. First posted on arXiv 27 May 2026.

Builds on

18 of 31 references.

References · 18

  1. 2019Language Models as Knowledge Bases?cited by 1,936
  2. 2023Steering Llama 2 via Contrastive Activation Additioncited by 1,085
  3. 2023Steering Language Models With Activation Engineeringcited by 922
  4. 2019Plug and Play Language Models: A Simple Approach to Controlled Text Generationcited by 799
  5. 2019CTRL: A Conditional Transformer Language Model for Controllable Generationcited by 762
  6. 2021The FLORES-101 Evaluation Benchmark for Low-Resource and Multilingual Machine Translationcited by 560
  7. 2021FUDGE: Controlled Text Generation With Future Discriminatorscited by 376
  8. 2021XL-Sum: Large-Scale Multilingual Abstractive Summarization for 44 Languagescited by 347
  9. 2020XCOPA: A Multilingual Dataset for Causal Commonsense Reasoningcited by 333
  10. 2023The Belebele Benchmark: a Parallel Reading Comprehension Dataset in 122 Language Variantscited by 311
  11. 2025AxBench: Steering LLMs? Even Simple Baselines Outperform Sparse Autoencoderscited by 218
  12. 2024Improving Steering Vectors by Targeting Sparse Autoencoder Featurescited by 92
6 more
  1. 2025SAEs Are Good for Steering -- If You Select the Right Featurescited by 78
  2. 2020Moving Down the Long Tail of Word Sense Disambiguation with Gloss-Informed Biencoderscited by 61
  3. 2023Codebook Features: Sparse and Discrete Interpretability for Neural Networkscited by 48
  4. 2025Steering off Course: Reliability Challenges in Steering Language Modelscited by 34
  5. 2021LMMS Reloaded: Transformer-based Sense Embeddings for Disambiguation and Beyondcited by 17
  6. 2025Multi-Sense Embeddings for Language Models and Knowledge Distillationcited by 3

Built on by

Not cited yet.