Do Reasoning Models Enhance Embedding Models?
On the map: LLM reasoning → Reinforcement learning for reasoning
Abstract (arXiv)
State-of-the-art embedding models are increasingly derived from decoder-only Large Language Model (LLM) backbones adapted via contrastive learning. Given the emergence of reasoning models trained via Reinforcement Learning with Verifiable Rewards (RLVR), a natural question arises: do enhanced reasoning translate to superior semantic representations when these models serve as embedding initializations? Contrary to expectation, our evaluation on MTEB and BRIGHT reveals a **null effect**: embedding models initialized from RLVR-tuned backbones yield no consistent performance advantage over their base counterparts when subjected to identical training recipes. To unpack this paradox, we introduce **H**ierarchical **R**epresentation **S**imilarity **A**nalysis (HRSA), a framework that decomposes similarity across representation, geometry, and function levels. HRSA reveals that while RLVR induces irreversible latent manifold's local geometry reorganization and reversible coordinate basis drift, it preserves the global manifold geometry and linear readout. Consequently, subsequent contrastive learning drives strong alignment between base- and reasoning-initialized models, a phenomenon we term **Manifold Realignment**. Empirically, our findings suggest that unlike Supervised Fine-Tuning (SFT), RLVR optimizes trajectories within an existing semantic landscape rather than fundamentally restructuring the landscape itself.
Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in Reinforcement learning for reasoning, one of 17 directions in LLM reasoning. Builds on 41 of its 51 references that are in the map. First posted on arXiv 29 Jan 2026.
Builds on
41 of 51 references.
Closest ancestors · 7
Same direction, newest first
- 2025The Path Not Taken: RLVR Provably Learns Off the Principals11 Nov
- 2025Language Models that Think, Chat Better24 Sep
- 2025The Surprising Effectiveness of Negative Reinforcement in LLM Reasoning2 Jun
- 2025Does Reinforcement Learning Really Incentivize Reasoning Capacity in LLMs Beyond the Base Model?18 Apr
- 2025DAPO: An Open-Source LLM Reinforcement Learning System at Scale18 Mar
- 2024Tulu 3: Pushing Frontiers in Open Language Model Post-Training22 Nov
- 2017Proximal Policy Optimization Algorithms20 Jul
Other references · 34
- 2021Training Verifiers to Solve Math Word Problemscited by 10,109
- 2024DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Modelscited by 9,379
- 2025Qwen3 Technical Reportcited by 8,820
- 2018Representation Learning with Contrastive Predictive Codingcited by 7,777
- 2021Measuring Mathematical Problem Solving With the MATH Datasetcited by 5,371
- 2013Efficient Estimation of Word Representations in Vector Spacecited by 4,375
- 2015Character-level Convolutional Networks for Text Classificationcited by 2,330
- 2021SimCSE: Simple Contrastive Learning of Sentence Embeddingscited by 2,173
- 2024MMLU-Pro: A More Robust and Challenging Multi-Task Language Understanding Benchmarkcited by 1,848
- 2019Similarity of Neural Network Representations Revisitedcited by 1,755
- 2018World Modelscited by 1,513
- 2025Qwen3 Embedding: Advancing Text Embedding and Reranking Through Foundation Modelscited by 1,261
22 more
- 2025Group Sequence Policy Optimizationcited by 835
- 2022MTEB: Massive Text Embedding Benchmarkcited by 640
- 2024NV-Embed: Improved Techniques for Training LLMs as Generalist Embedding Modelscited by 472
- 2024Generative Representational Instruction Tuningcited by 233
- 2025RL's Razor: Why Online Reinforcement Learning Forgets Lesscited by 176
- 2024BRIGHT: A Realistic and Challenging Benchmark for Reasoning-Intensive Retrievalcited by 158
- 2025MMTEB: Massive Multilingual Text Embedding Benchmarkcited by 125
- 2025Gemini Embedding: Generalizable Embeddings from Geminicited by 94
- 2022Reliability of CKA as a Similarity Measure in Deep Learningcited by 87
- 2024NV-Retriever: Improving text embedding models with effective hard-negative miningcited by 84
- 2024Are Your LLMs Capable of Stable Reasoning?cited by 65
- 2025LLM-JEPA: Large Language Models Meet Joint Embedding Predictive Architecturescited by 38
- 2024What Representational Similarity Measures Imply about Decodable Informationcited by 18
- 2024Embedding-Aligned Language Modelscited by 9
- 2025Tiny Model, Big Logic: Diversity-Driven Optimization Elicits Large-Model Reasoning Ability in VibeThinker-1.5Bcited by 8
- 2023Decoding In-Context Learning: Neuroscience-inspired Analysis of Representations in Large Language Modelscited by 7
- 2023Towards Measuring Representational Similarity of Large Language Modelscited by 6
- 2025Spectral Insights into Data-Oblivious Critical Layers in Large Language Modelscited by 4
- 2025Search-R3: Unifying Reasoning and Embedding in Large Language Modelscited by 4
- 2019Situating Sentence Embedders with Nearest Neighbor Overlapcited by 3
- 2025Reinforcement Learning Fine-Tunes a Sparse Subnetwork in Large Language Modelscited by 2
- 2025Cross-Model Semantics in Representation Learningcited by 2
Built on by
Not cited yet.
Similar, not cited
Close in wording, no citation either way
- 2026Beyond Alignment: Expanding Reasoning Capacity via Manifold-Reshaping Policy Optimization
- 2026MMEmb-R1: Reasoning-Enhanced Multimodal Embedding with Pair-Aware Selection and Adaptive Control
- 2026Embed-RL: Reinforcement Learning for Reasoning-Driven Multimodal Embeddings
- 2025Efficient Post-Training Refinement of Latent Reasoning in Large Language Models
- 2026Learning to Steer, Steering to See: Unveiling the Geometry of RLVR in Large Language Models via Trainable Vectors
- 2025Reinforcement Learning with Verifiable Rewards Implicitly Incentivizes Correct Reasoning in Base LLMs
- 2026Rotate2Think: Geometric Priming via Orthogonal Rotation to Improve Language Model Reasoning
- 2025Latent Reasoning via Sentence Embedding Prediction