Update on pacific island rainfall outlooks
Stuart Burgess, NIWA
More than 30 rainfall outlooks for the Southwest Pacific have been issued through the ICU, so we now have enough information to accurately assess how the forecasts have been faring over the past few years. Figure 1 indicates how often the forecasts have been correct, in a categorical sense. Taking the forecast category (below, average or below, average, average or above, or above average) to be that assigned the highest probability, the contours and numbers show the percentage frequency of correct forecasts (or the “hit” rate). The overall hit rate is equivalent to the percentage frequency of correct forecasts. While this is a simple (and probably optimistic) measure of skill for probability forecasts, it provides a reasonable indication of our skill for the available sample.
Since our earlier analysis (see ICU Issue No. 11), we have maintained a hit rate above 50% over much of our forecast area. The hit rate has been highest (>70%) in the 0-10°S latitudes west of about 155°W, in the region from Western Kiribati to the Northern Cook Islands, including Tokelau, and especially over Tuvalu (81%). However, scores have been much lower in areas around and to the south and east of Samoa.
Looking through time (see Figure 2), the average hit rate for the whole of the forecast region has ranged between about 45 and 80% throughout the period. If the forecasts were random guesses, we’d expect to get one of the single categories (below, average, or above average) correct one third of the time, and one of the composite categories (average or below, etc.) correct two thirds of the time. Hence, because we forecast overlapping and composite outcomes, a series of random forecasts would be correct half the time on average. Analysis of the statistics from the 30 rainfall outlooks overall shows that hit rates have exceeded 50% for the majority of our forecasts, indicating more skill than “chance” or climatology.
The distribution of forecast success may reflect to some extent the influence of ENSO events and the location of the SPCZ. The area near Samoa is constantly in the transition zone for SPCZ-related rainfall, making it a difficult region to predict. At this stage there is a slight indication that forecast confidence reaches a peak during the South Pacific summer, and falls to a minimum during the South Pacific autumn, but further statistics are required.
Figure 1. Frequency of correct 3-month rainfall outlooks. A score of 100 would mean that all were correct.
Figure 2. Validation score for 30 (3-month) rainfall outlooks. This chart shows the % of correct forecasts. Scores of 60% or more indicate significantly better outcomes than chance.