mlsanityData to 14 Oct 2026

One Ruler: A Same-Hands Re-Evaluation of Bivariate Causal Direction on Tuebingen, with a Parameter-Free Compression Baseline

arXiv 2606.23767 ↗First posted 22 Jun 2026cs.LGcited by 0

Not on the map. Why?

Abstract (arXiv)

Headline accuracies on the Tuebingen cause-effect pairs are routinely compared across papers even though each is measured under its authors' own protocol -- different pair subsets, weightings, model-selection, and decision rates. We argue this is the wrong comparison and run the right one: a same-hands re-evaluation in which every method is run by us on the identical 102 pairs, with one strict rule -- no tuning and a decision forced on every pair. As a clean reference point we introduce a deliberately minimal baseline: sorted-conditional compression, which feeds quantized, sorted, first-differenced data to an off-the-shelf compressor (bz2) and has zero fitted parameters. Under the common ruler the ranking differs sharply from the literature. Our baseline reaches 74.7% weighted accuracy (p = 3.7e-7); on the same 100 pairs that SLOPE is evaluated on it scores 76.0%, a 1.2-point gap below the authors' own forced-decision SLOPE (77.2%) that is well inside noise (McNemar p = 0.39). A faithful re-run of RECI lands at 70.7% -- inside the original authors' reported error bar, not the 77.5% often quoted (which we trace to a mis-copied cell). SLOPE's published 82.4% is a decided-subset figure: scoring the authors' own stored output only on the pairs its significance test chose to answer reproduces 81.7%. Under the common ruler the methods cluster in the low-to-mid 70s and the zero-parameter compressor ties the strongest of them. We document the mechanisms that inflate published figures (test-set model selection, significance-gated abstention) and contribute two further results: compression score magnitude is a model-free confounding flag (p = 2.8e-68), and a pre-registered falsification test fails in an instructive way that bounds the method's theoretical interpretation. Code, pre-registrations, and per-pair outputs are released.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 2 of its 7 references that are in the map. First posted on arXiv 22 Jun 2026.

Builds on

2 of 7 references.

References · 2

  1. 2017Telling Cause from Effect using MDL-based Local and Global Regressioncited by 21
  2. 2025Identifying Causal Direction via Dense Functional Classescited by 3

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2026CausalReasoningBenchmark: A Real-World Benchmark for Disentangled Evaluation of Causal Identification and Estimation
  2. 2025Compressed Causal Reasoning: Quantization and GraphRAG Effects on Interventional and Counterfactual Accuracy
  3. 2026CausalWrap: Model-Agnostic Causal Constraint Wrappers for Tabular Synthetic Data
  4. 2026CausalArena: Benchmarking Causal Discovery in the Foundation Model Era
  5. 2026Evidence-Type Competition: When Can Interventional Data Teach Language Models Causal Direction?
  6. 2026ReplaySCM: A Benchmark for Executable Causal Mechanism Induction from Interventions
  7. 2024CausalGraph2LLM: Evaluating LLMs for Causal Queries
  8. 2025Ice Cream Doesn't Cause Drowning: Benchmarking LLMs Against Statistical Pitfalls in Causal Inference