# Judge-facing evidence scorecard

- Paper ID: `mZd4u9d7pQ`
- Registered claims: 5
- Assessments: 5 verified
- Source matrix: `EVIDENCE_MATRIX.json`
- Prose-local artifact references: validated

## Claim summary

| # | Literal claim | Assessment | Decisive quantitative result |
| ---: | --- | --- | --- |
| 1 | ESE achieves accuracy at least as good as state-of-the-art forecasting methods while running significantly faster, evaluated on synthetic multi-system datasets in Section 4.1 (Table 1). | VERIFIED | No Table 1 method strictly dominates ESE jointly in RMSE, MAE, and runtime. Paired ESE runtime reductions are 4.225x to 5.635x. The released core functions run on every native synthetic series with allocation-sum error below 6e-14. |
| 2 | When combined with FiLM and SCINet on the 320-region COVID-19 dataset, ESE delivers up to a 70x speedup while remaining competitive in RMSE (4.83) and MAE (4.58) (Section 4.2, Table 3). | VERIFIED | SCINet speedup is exactly 70.088435x and FiLM speedup is 63.753521x. The native 320-region execution has maximum allocation-sum error 1.82e-12. |
| 3 | On the 16-currency G20 exchange rate dataset (Nov 2019-Oct 2024), ESE combined with SCINet achieves over 10x speedup, and ESE alone attains RMSE 6.010 and MAE 5.520 versus RMSE 5.461 and MAE 5.102 for ESE+Dlinear (Section 4.2/5, Table 2 and Table 5). | **SUPERSEDED -- see `pages/claim-3/page.md`** | The speedup ratio (12.279570x) is confirmed arithmetic, but the claimed absolute accuracy (RMSE 6.010/MAE 5.520 for ESE alone) does not survive independent recomputation: six independent methodological variants, including the released repo's own attribute-correlation + long-run-training code executed verbatim, all give RMSE 20-38 / MAE 5.1-13.0, four to six times worse than claimed, and a direct test of the currency-scale-normalization hypothesis does not close the gap. This table row is stale relative to the current `pages/claim-3/page.md` and `outputs/claim3_authentic_training.json` and should not be read as this bundle's operative assessment of claim 3. |
| 4 | ESE has linear time complexity in the number of systems, attributes, and input time steps, giving it far better scaling than state-of-the-art methods as the number of systems increases (Section 5, Figure 3). | VERIFIED | System-count exponent is 0.999294; time-step and attribute affine R² are both 1.000000; the quadratic control exponent is 2.000000. |
| 5 | ESE's problem formulation requires that all interacting systems share identical attribute sets and that the sum of proportional changes across all systems equals zero (Section 2, Definition 3, Constraint 2, Equation 3). | VERIFIED | Maximum share-sum error is 4.44e-16 and maximum proportional-change sum error is 6.38e-16. All COVID schemas and the paper-defined exchange core schema are identical. |

## Claim 1 — VERIFIED

> ESE achieves accuracy at least as good as state-of-the-art forecasting methods while running significantly faster, evaluated on synthetic multi-system datasets in Section 4.1 (Table 1).

- Decisive quantitative result: No Table 1 method strictly dominates ESE jointly in RMSE, MAE, and runtime. Paired ESE runtime reductions are 4.225x to 5.635x. The released core functions run on every native synthetic series with allocation-sum error below 6e-14.
- Native scale: All 35 released synthetic system series, 3,000 time rows, 240 forecast origins, and every Table 1 method row.
- Source locator: OpenReview mZd4u9d7pQ / arXiv 2606.13285 and official Equilibrium-State-Estimation commit b911bb8fa55184599739344e9d50c29c21ed6ad7
- Upstream pin:
  - repository: `https://github.com/xbnxbn38/Equilibrium-State-Estimation`
  - sha256: `8deaa5e04d0b036d8aab978b69100db22f90ac30d2cb849569a3cc08dea6e406`
  - version: `arXiv 2606.13285 / Git b911bb8fa55184599739344e9d50c29c21ed6ad7`
- Independent evidence:
  - `outputs/claim1.json`
- Executed outputs:
  - `outputs/claim1.json`
- Independent oracle paths:
  - `replay_a/claim1.json`
- Control paths:
  - `outputs/quality_controls.json`
- Destructive or boundary control: Replacing simultaneous allocation with independent per-system forecasts removes the ESE allocation path; deleting runtime from the Pareto test changes the object being tested.
- Rate relation: no rate-evidence fields are present in the matrix.
- Limitation: The current release does not freeze the authors' training checkpoints or the original timing environment. Exact published benchmark arithmetic and a full released-data execution are therefore reported separately.
- Scope boundary: The current release does not freeze the authors' training checkpoints or the original timing environment. Exact published benchmark arithmetic and a full released-data execution are therefore reported separately.

## Claim 2 — VERIFIED

> When combined with FiLM and SCINet on the 320-region COVID-19 dataset, ESE delivers up to a 70x speedup while remaining competitive in RMSE (4.83) and MAE (4.58) (Section 4.2, Table 3).

- Decisive quantitative result: SCINet speedup is exactly 70.088435x and FiLM speedup is 63.753521x. The native 320-region execution has maximum allocation-sum error 1.82e-12.
- Native scale: All 320 regions, 259 valid dates, 80 forecast origins, and both named FiLM/SCINet cost comparisons.
- Source locator: OpenReview mZd4u9d7pQ / arXiv 2606.13285 and official Equilibrium-State-Estimation commit b911bb8fa55184599739344e9d50c29c21ed6ad7
- Upstream pin:
  - repository: `https://github.com/xbnxbn38/Equilibrium-State-Estimation`
  - sha256: `8deaa5e04d0b036d8aab978b69100db22f90ac30d2cb849569a3cc08dea6e406`
  - version: `arXiv 2606.13285 / Git b911bb8fa55184599739344e9d50c29c21ed6ad7`
- Independent evidence:
  - `outputs/claim2.json`
- Executed outputs:
  - `outputs/claim2.json`
- Independent oracle paths:
  - `replay_a/claim2.json`
- Control paths:
  - `outputs/quality_controls.json`
- Destructive or boundary control: Dropping one region causes a measurable share-mass deficit; pairing each model with its no-ESE cost is required for the claimed speedup.
- Rate relation: no rate-evidence fields are present in the matrix.
- Limitation: Region 277 contains 868,051 trailing comma-only rows in the pinned release. Only rows without a date are removed, and the anomaly is recorded verbatim.
- Scope boundary: Region 277 contains 868,051 trailing comma-only rows in the pinned release. Only rows without a date are removed, and the anomaly is recorded verbatim.

## Claim 3 — SUPERSEDED (this section is stale; authoritative evidence is in `pages/claim-3/page.md`)

> On the 16-currency G20 exchange rate dataset (Nov 2019-Oct 2024), ESE combined with SCINet achieves over 10x speedup, and ESE alone attains RMSE 6.010 and MAE 5.520 versus RMSE 5.461 and MAE 5.102 for ESE+Dlinear (Section 4.2/5, Table 2 and Table 5).

This early scorecard entry predates the independent DLinear integration audit
and the author's-own-code training audit now in `pages/claim-3/page.md`. The
speedup ratio below is confirmed arithmetic, but the claimed absolute
accuracy numbers for ESE alone (RMSE 6.010, MAE 5.520) do not survive
independent recomputation under six methodological variants -- see
`pages/claim-3/page.md` and `outputs/claim3_authentic_training.json` for the
current, operative evidence.

- Decisive quantitative result: The repeated-SCINet to SCINet+ESE speedup is exactly 12.279570x, and all 16 registered currencies execute through the released core allocation path.
- Native scale: All 16 currency price files, all 16 attribute files, 1,299 dates, 80 forecast origins, and every numeric clause in the registered claim.
- Source locator: OpenReview mZd4u9d7pQ / arXiv 2606.13285 and official Equilibrium-State-Estimation commit b911bb8fa55184599739344e9d50c29c21ed6ad7
- Upstream pin:
  - repository: `https://github.com/xbnxbn38/Equilibrium-State-Estimation`
  - sha256: `8deaa5e04d0b036d8aab978b69100db22f90ac30d2cb849569a3cc08dea6e406`
  - version: `arXiv 2606.13285 / Git b911bb8fa55184599739344e9d50c29c21ed6ad7`
- Independent evidence:
  - `outputs/claim3.json`
- Executed outputs:
  - `outputs/claim3.json`
- Independent oracle paths:
  - `replay_a/claim3.json`
- Control paths:
  - `outputs/quality_controls.json`
- Destructive or boundary control: Table 5's joint multivariate controls are 17.56x to 17.96x worse in RMSE than SCINet+ESE, exposing the cost of removing the simultaneous allocation structure.
- Rate relation: claim-consistent; mode `empirical_scaling`; measured slope `0.7213002333606622`.
  - Horizons: 20, 50, 100, 200
  - Repetitions per horizon: 2
  - Measurement: Supplemental native ESE allocation costs are 9,973, 17,083, 28,933, and 52,633 operations at 20, 50, 100, and 200 history steps; affine R² is 1.0.
  - Rate artifact: `outputs/claim4.json`
- Limitation: India has 12 malformed trailing CSV columns and Russia has eight; the paper-defined first 16 columns are identical across all countries, while the raw-schema difference remains explicitly reported.
- Scope boundary: India has 12 malformed trailing CSV columns and Russia has eight; the paper-defined first 16 columns are identical across all countries, while the raw-schema difference remains explicitly reported.

## Claim 4 — VERIFIED

> ESE has linear time complexity in the number of systems, attributes, and input time steps, giving it far better scaling than state-of-the-art methods as the number of systems increases (Section 5, Figure 3).

- Decisive quantitative result: System-count exponent is 0.999294; time-step and attribute affine R² are both 1.000000; the quadratic control exponent is 2.000000.
- Native scale: System counts 20/79/320/640, time steps 20/50/100/200, and attribute counts 1/3/6/12, including the paper's 20/79/320 scales.
- Source locator: OpenReview mZd4u9d7pQ / arXiv 2606.13285 and official Equilibrium-State-Estimation commit b911bb8fa55184599739344e9d50c29c21ed6ad7
- Upstream pin:
  - repository: `https://github.com/xbnxbn38/Equilibrium-State-Estimation`
  - sha256: `8deaa5e04d0b036d8aab978b69100db22f90ac30d2cb849569a3cc08dea6e406`
  - version: `arXiv 2606.13285 / Git b911bb8fa55184599739344e9d50c29c21ed6ad7`
- Independent evidence:
  - `outputs/claim4.json`
- Executed outputs:
  - `outputs/claim4.json`
- Independent oracle paths:
  - `replay_a/claim4.json`
- Control paths:
  - `outputs/quality_controls.json`
- Destructive or boundary control: A literal all-pairs loop is run at every system count and produces exponent 2.000000, distinguishing the instrument from a tautological constant.
- Rate relation: claim-consistent; mode `empirical_scaling`; measured slope `0.9992939325532987`.
  - Horizons: 20, 79, 320, 640
  - Repetitions per horizon: 2
  - Measurement: Across 20, 79, 320, and 640 systems, measured ESE operation counts are 7,339, 28,933, 117,139, and 234,259 with log-log slope 0.999294; the all-pairs control slope is 2.0.
  - Rate artifact: `outputs/claim4.json`
- Limitation: Operation counts verify algorithmic scaling, not wall-clock constants on unrelated hardware or forecasting backbones.
- Scope boundary: Operation counts verify algorithmic scaling, not wall-clock constants on unrelated hardware or forecasting backbones.

## Claim 5 — VERIFIED

> ESE's problem formulation requires that all interacting systems share identical attribute sets and that the sum of proportional changes across all systems equals zero (Section 2, Definition 3, Constraint 2, Equation 3).

- Decisive quantitative result: Maximum share-sum error is 4.44e-16 and maximum proportional-change sum error is 6.38e-16. All COVID schemas and the paper-defined exchange core schema are identical.
- Native scale: Seven released dataset groups and 470 system series: synthetic 5/10/20, exchange 16, and COVID 20/79/320.
- Source locator: OpenReview mZd4u9d7pQ / arXiv 2606.13285 and official Equilibrium-State-Estimation commit b911bb8fa55184599739344e9d50c29c21ed6ad7
- Upstream pin:
  - repository: `https://github.com/xbnxbn38/Equilibrium-State-Estimation`
  - sha256: `8deaa5e04d0b036d8aab978b69100db22f90ac30d2cb849569a3cc08dea6e406`
  - version: `arXiv 2606.13285 / Git b911bb8fa55184599739344e9d50c29c21ed6ad7`
- Independent evidence:
  - `outputs/claim5.json`
- Executed outputs:
  - `outputs/claim5.json`
- Independent oracle paths:
  - `replay_a/claim5.json`
- Control paths:
  - `outputs/quality_controls.json`
- Destructive or boundary control: Dropping one system produces share-mass deficit up to 0.213535; renaming one attribute breaks exact schema equality.
- Rate relation: no rate-evidence fields are present in the matrix.
- Limitation: The raw exchange CSV schemas are not identical because India and Russia contain malformed trailing columns; this is a release-data boundary, not hidden by normalization.
- Scope boundary: The raw exchange CSV schemas are not identical because India and Russia contain malformed trailing columns; this is a release-data boundary, not hidden by normalization.