mlsanityData to 7 Oct 2026

Reading a Legal Question Word by Word: Embedding Trajectories of 2,144 Vietnamese Legal Headlines

arXiv 2609.08372 ↗First posted 8 Sep 2026cs.CLcited by 0

Not on the map. Why?

Abstract (arXiv)

A dense retriever encodes a question as one vector, but the question arrives one word at a time. We read 2,144 held-out headlines from Thu Vien Phap Luat (Vietnamese legal library) word by word with Nemotron-3-Embed 8B/1B and Qwen3-Embedding 8B/0.6B, encoding 65,444 prefixes against 20,034 articles, plus every prefix of 3,438 sub-questions from 1,112 multi-question headlines and of 168 answers. (i) The gold article becomes rank 1 after a median of 6-7 content words in every encoder, before the interrogative frame is read, and stays there to the end in 78-85% of cases. (ii) In a multi-question headline the lock is inside the first sub-question 94-98% of the time; the second leaves rank unchanged in 89-95%; encoded alone, the second reaches rank 1 in 42-58% vs 91-96% for the first, at the same lock word (95-97% identical). (iii) Numbers, dates and instrument identifiers move the embedding twice as far as content words and four times as far as interrogative words; 72-78% of steps move toward the gold article, and the closing interrogative frame moves against that direction in 95-99% of headlines. (iv) Rank/cosine clustering yields six archetypes (instant, typical, unstable, late, never-locking) that differ by legal area and form (chi-squared p < 1e-8): real-estate and litigation headlines never lock on a number; environmental and accounting headlines do so a third of the time. (v) An answer read word by word retrieves its article after 8-16 words and addresses the sub-questions in order asked in 83-89% of cases. (vi) A word's step keeps a consistent direction across headlines (cosine 0.25-0.33; 0.44-0.60 for numbers); a preceding question rotates that step by about 60 degrees and a greeting by about 30 degrees; steps shrink as i^{-0.8}; and a two-question headline is within 12-17 degrees of a linear mix of its two questions. We call this a context-modulated additive walk.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 20 of its 41 references that are in the map. First posted on arXiv 8 Sep 2026.

Builds on

20 of 41 references.

References · 20

  1. 2019Sentence-BERT: Sentence Embeddings using Siamese BERT-Networkscited by 9,049
  2. 2020Dense Passage Retrieval for Open-Domain Question Answeringcited by 3,779
  3. 2019Similarity of Neural Network Representations Revisitedcited by 1,695
  4. 2025Qwen3 Embedding: Advancing Text Embedding and Reranking Through Foundation Modelscited by 1,175
  5. 2019How Contextual are Contextualized Word Representations? Comparing the Geometry of BERT, ELMo, and GPT-2 Embeddingscited by 770
  6. 2024LLM2Vec: Large Language Models Are Secretly Powerful Text Encoderscited by 396
  7. 2019A Multiscale Visualization of Attention in the Transformer Modelcited by 292
  8. 2019Representation Degeneration Problem in Training Natural Language Generation Modelscited by 250
  9. 2019Visualizing and Measuring the Geometry of BERTcited by 215
  10. 2022SGPT: GPT Sentence Embeddings for Semantic Searchcited by 204
  11. 2021All Bark and No Bite: Rogue Dimensions in Transformer Language Models Obscure Representational Qualitycited by 150
  12. 2020The Language Interpretability Tool: Extensible, Interactive Visualizations and Analysis for NLP Modelscited by 125
8 more
  1. 2024NV-Retriever: Improving text embedding models with effective hard-negative miningcited by 83
  2. 2016The emotional arcs of stories are dominated by six basic shapescited by 69
  3. 2018VnCoreNLP: A Vietnamese Natural Language Processing Toolkitcited by 51
  4. 2017A Fast and Accurate Vietnamese Word Segmentercited by 21
  5. 2022A Summary of the ALQAC 2021 Competitioncited by 9
  6. 2015Network Analysis in the Legal Domain: A complex model for European Union legal sourcescited by 3
  7. 2022LegalVis: Exploring and Inferring Precedent Citations in Legal Documentscited by 3
  8. 2022Multi-stage Information Retrieval for Vietnamese Legal Textscited by 3

Built on by

Not cited yet.