The fractions skill score for ensemble forecast verification

Author(s)
Tobias Necker, Ludwig Wolfgruber, Lukas Kugler, Martin Weissmann, Manfred Dorninger, Stefano Serafin
Abstract

The fractions skill score (FSS) is a neighbourhood verification method originally designed to verify deterministic forecasts of binary events. Previous studies employed different approaches for computing an ensemble-based FSS for probabilistic forecast verification. We show that the formulation of an ensemble-based FSS substantially affects verification results. Comparing four possible approaches, we determine how different ensemble-based FSS variants depend on ensemble size, neighbourhood size, and forecast event frequency of occurrence. We demonstrate that only one ensemble-based FSS, which we call the probabilistic FSS (pFSS), is well behaved and reasonably dependent on ensemble size. Furthermore, we derive a relationship to describe how the pFSS behaves with ensemble size. The proposed relationship is similar to a known result for the Brier skill score. Our study uses high-resolution 1000-member ensemble precipitation forecasts from a high-impact weather period. The large ensemble enables us to study the influence of ensemble and neighbourhood size on forecast skill by deriving probabilistic skilful spatial scales.

Organisation(s)
Department of Meteorology and Geophysics
Journal
Quarterly Journal of the Royal Meteorological Society
Volume
150
Pages
4457-4477
ISSN
0035-9009
DOI
https://doi.org/10.1002/qj.4824
Publication date
2024
Peer reviewed
Yes
Austrian Fields of Science 2012
105206 Meteorology
Keywords
ASJC Scopus subject areas
Atmospheric Science
Portal url
https://ucrisportal.univie.ac.at/en/publications/the-fractions-skill-score-for-ensemble-forecast-verification(ff9b5f07-a319-406f-886a-9203d550fe53).html