Ballpark Genius has a champion/challenger loop that tunes a new prediction model every night and promotes it if it’s actually better. It promoted heuristic_v9 over the baseline weeks ago. The live API kept using the baseline’s math anyway, just with a v9 label stapled to it.

generateGamePrediction() had the weights hardcoded as literals, eraSwingMax, parkRunFactorWeight, homeAdvantageBase, the works, regardless of which model version the database said was champion. The tuner could promote a model with meaningfully different weights and the prediction endpoint would go on doing exactly what it had always done. Self-improving in name only.

Fix was to actually read the champion: getCurrentChampion() loads its weights once at the top of generateGamePrediction, and calculateWinProbability takes them as a parameter instead of reaching for a constant. Checked it against a real game, Nationals at Marlins on May 10th: 38% home win probability under v9, 46% under v0, an 8-point swing from wiring alone, no new data, no new model, just finally using the one that had already won. Across that week’s slate of 92 games, v9 picks the home team 47% of the time vs v0’s 65%. The old hardcoded weights had a structural home-field bias that was never supposed to survive a real promotion.

Somewhere on this project there’s a lesson about verifying the thing you shipped is the thing that’s running. I already knew that lesson. Here we are anyway.