Every value in topos/evaluation/policies/calibration.py is downstream of the evidence below. This report re-syncs the
live leaderboard corpus, shows the distribution it produces, and derives the calibration
values that distribution now recommends. Add more packages and languages and the picture
— and the thresholds it justifies — sharpen.
Every file in the cohort is evaluated by Topos and recorded in the leaderboard corpus.
Per-metric histograms and per-dimension ECDFs reveal where typical code ends and outliers begin.
Elbows and percentiles recommend the gates & floors committed to calibration.py.
Topos applies those values; generator achievements meet on the lattice to award a medal.
This snapshot drives everything downstream. It grows along two axes — more packages and more languages — and each addition tightens the distribution.
Language coverage: python (34371), javascript (10473), rust (9779). Packages such as @babel/compat-data, @babel/core, @babel/helper-compilation-targets, @babel/helper-define-polyfill-provider, @babel/helper-globals, @babel/helper-module-imports, @babel/helper-module-transforms, @babel/helpers….
Topos thresholds are tuned on multi-ecosystem leaderboard cohorts: each file is evaluated with Topos probes, ecosystems are equal-strata sampled so Python cannot dominate, raw metric gates are set from distribution percentiles, and normalized score floors are chosen from ECDF elbows and pass-rate analysis.
The distribution above is the input; these are the numbers it implies for
calibration.py. The committed column is parsed live from the policy module so drift is visible at a glance.
Could not load committed values — showing recommendations only. Source: topos v0.4.3 · calibration.py ↗.
(SCORE_FLOORS)| Generator | Recommended | Committed | Drift | Basis (from corpus) |
|---|---|---|---|---|
| Simple | 0.95 | — | no committed value | ECDF elbow at 0.97 → rounded to 0.95. |
| Composable | 0.95 | — | no committed value | ECDF elbow at 0.97 → rounded to 0.95. |
| Secure | 1 | — | no committed value | Categorical security — held at 1.00 regardless of elbow. |
| Gate | Committed bound | Corpus evidence | Why |
|---|---|---|---|
| McCabe cyclomatic complexity SIMPLE.max_cyclomatic |
<= 15 | median=—, p75=23 | Upper bound near the corpus median — most files stay below it. |
| Max single-function complexity SIMPLE.max_function_complexity |
<= 10 | median=—, p75=9 | Caps the worst function around the corpus 75th percentile. |
| Kolmogorov AST entropy band SIMPLE.min_entropy / max_entropy |
[0.2, 0.8] | p5=0.16, p95=0.67 | Healthy band spans the bulk of the corpus (p5–p95). |
| Martin module instability band COMPOSABLE.instability_low / instability_high |
[0.3, 0.7] | p25=0.5, p75=1 | Centred on the corpus median (≈0.5) — balanced coupling. |
| Module fan-in COMPOSABLE.max_fan_in |
<= 15 | p75=0, p95=0 | Bound sits near the corpus 95th percentile. |
| Module fan-out COMPOSABLE.max_fan_out |
<= 15 | p75=0, p95=1 | Bound sits near the corpus 95th percentile. |
| Dangerous CPG calls SECURE.max_dangerous_calls |
== 0 | median=—, p95=1 | Categorical: the corpus median is 0 — any call fails. |
| Active taint flows SECURE.max_taint_flows |
== 0 | median=—, p95=0 | Categorical: zero-tolerance dataflow safety. |
Recalibration trigger: Elbow = argmax of |d²P/ds²| on the file-level ECDF (per dimension). Re-pick a threshold when the elbow differs from the current floor by more than 0.05.
Topos applies the calibrated gates and floors to each file. The three generator results — Simple, Composable, Secure — meet on an 8-element lattice; the lattice element maps to a medal.
| Lattice element | Medal | Files | |
|---|---|---|---|
IDEAL |
🥇 GOLD | 10390 | |
COMPOSABLE_SECURE |
🥈 SILVER | 4363 | |
SIMPLE_COMPOSABLE |
🥈 SILVER | 230 | |
SIMPLE_SECURE |
🥈 SILVER | 9360 | |
COMPOSABLE |
🥉 BRONZE | 413 | |
SECURE |
🥉 BRONZE | 3719 | |
SIMPLE |
🥉 BRONZE | 329 | |
SLOP |
❌ SLOP | 533 |