How good is the forecast?

Every forecast SwimSignal issues is logged and scored later against what the overflows actually did. This page updates itself. It also shows the offline tests the model was built on.

Reading the numbers

The Brier score is the mean squared difference between a probability and what happened (0 or 1). Lower is better; 0 is perfect. Climatology means always forecasting the long-run average. Skill is how far below climatology's Brier score the forecast gets, as a fraction: 0 is no better than the average, 1 is perfect. AUC is how well the forecast ranks spill days above dry days: 0.5 is random, 1 is perfect. A reliability table checks that days forecast at 30% happen about 30% of the time.