mlsanityData to 14 Oct 2026

No Training Required: Exploring Random Encoders for Sentence Classification

arXiv 1901.10444 ↗First posted 29 Jan 2019cs.CLcited by 32

Not on the map. Why?

Abstract (arXiv)

We explore various methods for computing sentence representations from pre-trained word embeddings without any training, i.e., using nothing but random parameterizations. Our aim is to put sentence embeddings on more solid footing by 1) looking at how much modern sentence embeddings gain over random methods---as it turns out, surprisingly little; and by 2) providing the field with more appropriate baselines going forward---which are, as it turns out, quite strong. We also make important observations about proper experimental protocol for sentence classification evaluation, together with recommendations for future research.

Key facts. Cited by 32 papers in the map (data through 14 Oct 2026). First posted on arXiv 29 Jan 2019.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2021: 12; 2022: 6; 2023: 2; 2024: 4; 2025: 4; 2026: 4.

32 papers.

Sort:

By area:

  1. 2021Masked Language Modeling and the Distributional Hypothesis: Order Word Matters Pre-training for Littlecited by 177
  2. 2021State Entropy Maximization with Random Encoders for Efficient Explorationcited by 108
  3. 2021COM2SENSE: A Commonsense Reasoning Benchmark with Complementary Sentencescited by 40
  4. 2023Emergence of Maps in the Memories of Blind Navigation Agentscited by 33
  5. 2021An Isotropy Analysis in the Multilingual BERT Embedding Spacecited by 25
  6. 2024Mechanistic?cited by 25
  7. 2025Knowing Before Saying: LLM Representations Encode Information About Chain-of-Thought Success Before Completioncited by 23
  8. 2021Efficient and Private Federated Learning with Partially Trainable Networkscited by 13
  9. 2022Knowledge Distillation of Russian Language Models with Reduction of Vocabularycited by 7
  10. 2022Empirical Evaluation and Theoretical Analysis for Representation Learning: A Surveycited by 5
  11. 2023Backdoor Learning for NLP: Recent Advances, Challenges, and Future Research Directionscited by 4
  12. 2025Identifying and Mitigating the Influence of the Prior Distribution in Large Language Modelscited by 4
20 more
  1. 2021On the Interpretability of Deep Learning Based Models for Knowledge Tracingcited by 3
  2. 2021What's Hidden in a One-layer Randomly Weighted Transformer?cited by 3
  3. 2024Private Language Models via Truncated Laplacian Mechanismcited by 3
  4. 2021Cost-effective Deployment of BERT Models in Serverless Environmentcited by 2
  5. 2025Neural network embeddings recover value dimensions from psychometric survey items on par with human datacited by 2
  6. 2021Sentence Embeddings by Ensemble Distillationcited by 1
  7. 2021Text Classification and Clustering with Annealing Soft Nearest Neighbor Losscited by 1
  8. 2021General Cross-Architecture Distillation of Pretrained Language Models into Matrix Embeddingscited by 1
  9. 2022EigenNoise: A Contrastive Prior to Warm-Start Representationscited by 1
  10. 2024Text Classification using Graph Convolutional Networks: A Comprehensive Surveycited by 1
  11. 2026Emergent Manifold Separability during Reasoning in Large Language Modelscited by 1
  12. 2021Random Embeddings and Linear Regression can Predict Protein Functioncited by 0
  13. 2022Identifying stimulus-driven neural activity patterns in multi-patient intracranial recordingscited by 0
  14. 2022To Know by the Company Words Keep and What Else Lies in the Vicinitycited by 0
  15. 2022Probing of Quantitative Values in Abstractive Summarization Modelscited by 0
  16. 2024Establishing Deep InfoMax as an effective self-supervised learning methodology in materials informaticscited by 0
  17. 2025Syntactic Learnability of Echo State Neural Language Models at Scalecited by 0
  18. 2026Encoded but Not Actionable: Auditing the Decode-Generate-Steer Gap in Frozen LLMs for Geometric Constraintscited by 0
  19. 2026A Function-Level Vulnerability Score Measures Flag Rate More Than the Model: Protocol Effects on Paired Benchmarkscited by 0
  20. 2026How High Is 0.6? Floors, Ceilings, and Headroom in Interpretability Probingcited by 0