Attempt 2 scored 6 of 6. We’re rejecting the pass.

What ran: the two changes we committed to, nothing else

Quick recap. Our free frost lookup serves an elevation-corrected frost map that passed its own published test — Stage 1, in our build plan's naming. Stage 2 is the experimental add-on that reads terrain shape to predict cold-air pooling: the frost pockets Stage 1 can't see. Stage 2 failed its first gate — the gate being the pass/fail line we published before running anything — going 5 of 6 on the pre-registered criteria, with Cache Valley at 42% of a real 45-day frost-season gap against a declared 50% line. The failure analysis named two specific fixes, and we pre-registered both inside that results post, before this run existed:

• A within-pool height term. How high a spot sits above its cold pool's floor — the Yoshino variable from the cold-pool literature — so the model can tell a ledge partway up the bowl from the bottom of the bowl.
• A tail-aware correction. Replace the single all-nights scale-down factor (0.655) with per-threshold weighting toward the nights frost dates actually get set on.

Attempt 2 is those two changes. Same six criteria, same operationalizations (the exact numeric meanings of the fuzzy words) frozen in the pipeline code before attempt 1 ever ran, same hidden-station design — the model is always graded on stations hidden from its fitting — same two named cases, same clamps (the pre-set limits on how far any correction can move a date). Every constant in the new terms was fixed before any attempt-2 fit ran, and attempt 1's results were re-verified byte-identical after the shared code was parameterized. The bar did not move.

Here's what the bar caught.

Both new mechanisms died on contact with the data

The height term came back zero. The fitted coefficient — the strength the data assigns the new dial — came out at +0.0008 ± 0.0083 °C per meter. Not small — statistically zero. The textbook value we seeded it with (+0.0305 on the same scale, from the thermal-belt literature) is rejected by our own calibration data at about 3.6 standard errors — far outside what chance would explain.

The frustrating part is that the signal exists exactly where attempt 1's failure analysis said it would. The USU bench, 99 m up its pool, ran +1.94 °C warm in attempt 1's leftover errors (its residuals). Richmond, 79 m up, +2.15. Heber, 48 m up, +1.30. But box-wide, other stations cancel it. Pineview Dam sits 55 m up its pool and runs −4.33 °C — it's beside a drainage constriction where cold air jets through, the exact site type attempt 1 flagged. Deer Creek Dam, same story at −4.88. And Preston, Idaho reads as 82 m "up the pool" only because the Bear River cut a trench below it; it actually frosts with the plain it sits on.

Plain version: being partway up the bowl really does keep some places warmer. But our measurement of "partway up the bowl" also fires at dam outlets and above river trenches, where the same number means something completely different. Average the true signal with the false ones and you get zero.

The tail conjecture was falsified outright — by measurement. The whole idea rested on an assumption: frost dates get set disproportionately on calm, clear nights, the nights cold pooling happens. We finally measured it. They don't. Across station-years in our box, the nights that actually end a season are short on calm-clear conditions: about 13% of them are calm-clear (13.9% at 28 °F, 12.6% at 32 °F, 14.1% at 36 °F), versus 23% of all spring-and-fall nights. The opposite of the hypothesis.

So the tail-aware factor came out at 0.615 at 32 °F — slightly below the flat 0.655 it replaced. The pre-registered mechanism, as operationalized, is a measured no-op. Slightly contractionary, even.

We checked whether the measuring method was broken rather than the idea. It isn't. An independent identity — combine the measured calm-clear frequency with the measured shrinkage on the other nights, and you should recover the old flat factor — predicts 0.660. The independently fitted value is 0.655. The estimator is sound; the surprise is real. One disclosure: the first cut used a proxy (calm-clear frequency among nights near each threshold), it showed no enrichment either, and the switch to the literal pre-registered reading happened before any gate criterion was computed. Both numbers are in the full report.

Plain version: in Utah, the nights that actually end your growing season are mostly not the still, starry ones. They're the nights behind a cold front — or nights our clear-sky classifier mislabels, and the data can't yet tell us which. Either way, the assumption is dead, and we only know that because we wrote it down before checking.

That second death is the most useful number this run produced. A negative result you measured on purpose beats a positive one you assumed.

The gate: 6 of 6 by the frozen letter

Both new mechanisms dead, you'd expect this gate to fail harder than attempt 1's. Instead:

(Same reading rules as last time: "held out" and "leave-pair-out" mean the model is graded only on stations or pairs hidden from its fitting. MAE is the average miss in days. "Skill" is the fraction of Stage 1's error removed. "p10–p90 coverage" is the share of real outcomes inside the tool's 10–90% date range.)

Gate computation, by the frozen letter: all six pass. Our determination: NO-GO. Nothing ships. The next section is the difference between those two sentences.

Why we're throwing the score out

One piece of machinery first, because the whole verdict turns on it. We grade the model by cross-validation, which runs in rounds called folds: each fold hides a different station (or pair), refits the model on the rest, and grades it on what it couldn't see. And the fitting convention — frozen in attempt 1, unchanged here — says: a term whose fitted sign contradicts the physics, weakly, gets replaced by its standard value from the published literature. Call that rule the sign guard. In attempt 1 the guard fired identically in every fold. Three terms, always substituted, stable. Honest cross-validation.

The new height term broke that convention in a way nobody designed for. Its fit sits exactly at zero, so tiny changes in which stations are held out flip it across the sign boundary. Fitted at +0.0008, the model keeps +0.0008. Fitted at −0.0008 — statistically the same number — the guard silently swaps in +0.0305, the textbook default, 38 times larger. A cliff where there should be a slope.

Four of the 66 folds — one per hidden station — land on the wrong side of that cliff: precisely the folds that delete one of the warm, high-on-the-pool stations the term needs. So does the separate two-station fold that grades Cache Valley. Hold out the USU bench and the Trenton floor to test them, and the guard installs the textbook coefficient — which then "predicts" the bench beautifully: 39.5 of the 45 days. The textbook value happens to be nearly exactly right at USU. The box-wide data still rejects applying it everywhere. A model that only works when its most informative stations are deleted from the fit isn't a model — it's a prior wearing a validation costume.

And the shipping model — the one the map itself would actually be baked from, fit on all the data — keeps the coefficient at zero. On Cache Valley it reproduces 20.5 days of the 45. That's 45.6%. The declared line is 50%. Below it.

Plain version: the test-taking copy of the model and the shipping copy of the model were not the same model. Only the test-taking copy passed. Publishing its grade while serving its sibling would make the gate a prop.

Same artifact, smaller stakes, disclosed anyway: the one held-out pair that improved from wrong to right versus attempt 1 (Cutler Dam vs. Logan USU) also flipped inside one of those default-regime folds. Criterion 1 clears its bar without it either way — 96.2% before, and the bar is 90 — but you should know where the improvement came from.

The real movement, artifact-free

Strip the fold artifact out and here is what attempt 2 actually scored:

• Cache Valley, full fit: 19.0 → 20.5 days captured. 42.2% → 45.6%. A real gain of 1.5 days, where 3.5 were needed to reach the line.
• North Ogden vs. Eden: 38.0 days vs. attempt 1's 38.5 — fold-stable, no default regime involved, a real and slightly smaller pass.
• Complex-terrain error: 5.82 → 5.89 days. Marginally worse, still above the skill bar.
• Peter Sinks (diagnostic, not a gate point): still wrong-signed at +1.18 °C, vs. +1.26 in attempt 1. At 100 m resolution the sink's tiny floor barely exists, so the new term never engages there.

Progress on the one criterion that failed. Not close to enough, and everything else roughly flat or a hair worse. That's the honest score of attempt 2: 45.6%, not 6 of 6.

The stop: no attempt 3 tonight

The attempt loop we committed to before attempt 1 has a stop clause: instability means rethink, not retry. This is instability in the plainest sense: the decisive criterion’s pass depends on which stations happen to be held out, and the mechanism the pre-registration bet on was falsified outright. Re-running variations until one squeaks through is the exact behavior pre-registration exists to prevent — so the loop stops here, even though it had authorization for more runs tonight.

The redesign candidates, named but deliberately not pre-registered here:

• Replace the 0-or-everything sign guard with continuous shrinkage toward the literature value, so thin evidence pulls a coefficient partway instead of teleporting it 38×. That removes this artifact class entirely, for every term.
• Respecify the pool-height variable on terrain-only evidence: a floor definition that isn't fooled by river trenches, and a distinction between pool interiors and the drainage outlets that currently poison the fit.
• Retire or rebuild the tail hypothesis. If rebuilt, the nighttime-cloud classifier — the piece that labels which nights were actually clear — gets validated first, separately, before it touches this model.

None of that is a commitment yet. Naming next steps in the same post as the numbers they'd respond to is exactly how bars quietly move. The next pre-registration gets its own post — after the design work, before any run, same six criteria, same frozen operationalizations.

Why refuse your own passing grade

Publishing a failure, like last time, is the easy half of pre-registration. The hard half is this one: the scoreboard says 6 of 6, the fine print says the decisive point came from a crack in the test harness, and the rules you published leave you nothing to do but say so. A pass the shipped model can't reproduce isn't a pass. That sentence is the entire reason the bar was written down before the data existed. And behind the procedure there's a plainer reason. It sucks when plants die. It's worse when they die because a map told you they were safe.

Nothing changes for you. The free frost lookup keeps serving the Stage-1 elevation-corrected surface that passed its own published bar. No cold-air correction ships anywhere — not on the tool, and not in Colorado, Pennsylvania, or Oregon — the regions the correction was next planned to expand to. That planned rollout stays on hold.

When there's a redesigned model worth testing, its bar gets published here first, before the run. Then the numbers land here, either way. Same as this time.