Reading a Legal Question Word by Word: Embedding Trajectories of 2,144 Vietnamese Legal Headlines
Not on the map. Why?
Abstract (arXiv)
A dense retriever encodes a question as one vector, but the question arrives one word at a time. We read 2,144 held-out headlines from Thu Vien Phap Luat (Vietnamese legal library) word by word with Nemotron-3-Embed 8B/1B and Qwen3-Embedding 8B/0.6B, encoding 65,444 prefixes against 20,034 articles, plus every prefix of 3,438 sub-questions from 1,112 multi-question headlines and of 168 answers. (i) The gold article becomes rank 1 after a median of 6-7 content words in every encoder, before the interrogative frame is read, and stays there to the end in 78-85% of cases. (ii) In a multi-question headline the lock is inside the first sub-question 94-98% of the time; the second leaves rank unchanged in 89-95%; encoded alone, the second reaches rank 1 in 42-58% vs 91-96% for the first, at the same lock word (95-97% identical). (iii) Numbers, dates and instrument identifiers move the embedding twice as far as content words and four times as far as interrogative words; 72-78% of steps move toward the gold article, and the closing interrogative frame moves against that direction in 95-99% of headlines. (iv) Rank/cosine clustering yields six archetypes (instant, typical, unstable, late, never-locking) that differ by legal area and form (chi-squared p < 1e-8): real-estate and litigation headlines never lock on a number; environmental and accounting headlines do so a third of the time. (v) An answer read word by word retrieves its article after 8-16 words and addresses the sub-questions in order asked in 83-89% of cases. (vi) A word's step keeps a consistent direction across headlines (cosine 0.25-0.33; 0.44-0.60 for numbers); a preceding question rotates that step by about 60 degrees and a greeting by about 30 degrees; steps shrink as i^{-0.8}; and a two-question headline is within 12-17 degrees of a linear mix of its two questions. We call this a context-modulated additive walk.
Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 20 of its 41 references that are in the map. First posted on arXiv 8 Sep 2026.
Builds on
20 of 41 references.
References · 20
- 2019Sentence-BERT: Sentence Embeddings using Siamese BERT-Networkscited by 9,049
- 2020Dense Passage Retrieval for Open-Domain Question Answeringcited by 3,779
- 2019Similarity of Neural Network Representations Revisitedcited by 1,695
- 2025Qwen3 Embedding: Advancing Text Embedding and Reranking Through Foundation Modelscited by 1,175
- 2019How Contextual are Contextualized Word Representations? Comparing the Geometry of BERT, ELMo, and GPT-2 Embeddingscited by 770
- 2024LLM2Vec: Large Language Models Are Secretly Powerful Text Encoderscited by 396
- 2019A Multiscale Visualization of Attention in the Transformer Modelcited by 292
- 2019Representation Degeneration Problem in Training Natural Language Generation Modelscited by 250
- 2019Visualizing and Measuring the Geometry of BERTcited by 215
- 2022SGPT: GPT Sentence Embeddings for Semantic Searchcited by 204
- 2021All Bark and No Bite: Rogue Dimensions in Transformer Language Models Obscure Representational Qualitycited by 150
- 2020The Language Interpretability Tool: Extensible, Interactive Visualizations and Analysis for NLP Modelscited by 125
8 more
- 2024NV-Retriever: Improving text embedding models with effective hard-negative miningcited by 83
- 2016The emotional arcs of stories are dominated by six basic shapescited by 69
- 2018VnCoreNLP: A Vietnamese Natural Language Processing Toolkitcited by 51
- 2017A Fast and Accurate Vietnamese Word Segmentercited by 21
- 2022A Summary of the ALQAC 2021 Competitioncited by 9
- 2015Network Analysis in the Legal Domain: A complex model for European Union legal sourcescited by 3
- 2022LegalVis: Exploring and Inferring Precedent Citations in Legal Documentscited by 3
- 2022Multi-stage Information Retrieval for Vietnamese Legal Textscited by 3
Built on by
Not cited yet.